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Preface 



Diagrams 2000 is dedicated to the memory of Jon Barwise. 

Diagrams 2000 was the first event in a new interdisciplinary conference series 
on the Theory and Application of Diagrams. It was held at the University of 
Edinburgh, Scotland, September 1-3, 2000. 

Driven by the pervasiveness of diagrams in human communication and by 
the increasing availability of graphical environments in computerized work, the 
study of diagrammatic notations is emerging as a research field in its own right. 
This development has simultaneously taken place in several scientific disciplines, 
including, amongst others: cognitive science, artificial intelligence, and computer 
science. Consequently, a number of different workshop series on this topic have 
been successfully organized during the last few years: Thinking with Diagrams, 
Theory of Visual Languages, Reasoning with Diagrammatic Representations, 
and Formalizing Reasoning with Visual and Diagrammatic Representations. 

Diagrams are simultaneously complex cognitive phenonema and sophisti- 
cated computational artifacts. So, to be successful and relevant the study of 
diagrams must as a whole be interdisciplinary in nature. Thus, the workshop 
series mentioned above decided to merge into Diagrams 2000, as the single in- 
terdisciplinary conference for this exciting new held. It is intended that Diagrams 
2000 should become the premier international conference series in this area and 
provide a forum with sufficient breadth of scope to encompass researchers from 
all academic areas who are studying the nature of diagrammatic representations 
and their use by humans and in machines. 

The call for papers and posters for Diagrams 2000 attracted submissions 
from a very wide variety of disciplines and departments, including: architecture, 
art and design, artificial intelligence, cognitive science, computer science, edu- 
cation, engineering, human computer interaction, information science, manage- 
ment, mathematics, medicine, philosophy, psychology, speech pathology, textile 
technology. Submissions were received from countries all over the world, includ- 
ing: Austria, Australia, Denmark, Canada, Finland, France, Germany, Japan, 
Israel, Italy, New Zealand, The Netherlands, Poland, The USA, Spain, Switzer- 
land, and The UK. 

The standard of the accepted papers was high, with an acceptance rate of 
about 30%. The papers covered a wide variety of topics and for the sake of 
imposing some organizational structure on the conference, the presented papers 
were classified into the following themes: logic and diagrams; theoretical concerns 
about diagrams; cognition and diagrams; human communication and diagrams; 
diagrammatic reasoning and proof systems; diagrams to support the develop- 
ment of software systems, and systems to support the development of diagrams. 
Cutting across these themes was a substantial variety of different types of dia- 
grams. These ranged from classes of diagrams that are ubiquitous in this area 
of research (such as node-link formats, Euler/ Venn diagrams, bar charts, design 
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layouts) through to more specialized forms of diagrams for particular purposes 
(e.g., representations of time, Celtic Knots). 

In addition to the 31 technical paper presentations, in eight sessions, Di- 
agrams 2000 included: an invited talk by Alan MacEachren (Representations 
to mediate geospatial collaborative reasoning: A cognitive-semiotic perspective); 
a talk in memory of Jon Barwise given by Keith Stenning; a tutorial by Kim 
Marriott on formal approaches to visual languages; a tutorial on cognitive ap- 
proaches to diagrams co-presented by David Gooding, Hermi Sclrijf, and Jiajie 
Zhang; a session at which ten posters were presented. 

The program co-chairs would like to thank all the members of the program 
committee for all their efforts towards making Diagrams 2000 a success. We 
are particularly grateful to Jo Calder, Alan Blackwell, Bernd Meyer, and Nigel 
Birch. 
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Invited Talk: 

Representations to Mediate Geospatial 
Collaborative Reasoning: A Cognitive-Semiotic 

Perspective 



Alan M. MacEachren 



Geo VISTA Center (www.geovista.psu.edu), Department of Geography, 
Penn State University, USA, 

alanOgeog . psu . edu 

www . geovista . psu . edu/MacEachren/ alanhome . html 



Abstract. This presentation will address the representation of geospa- 
tial information in the context of group work. The focus is on visual 
representations that mediate between human collaborators who are par- 
ticipating in a joint reasoning process, within a place and/or space-based 
problem context. The perspective developed for addressing the challenges 
involved builds upon the cognitive-semiotic approach outlined in How 
Maps Work, extending it to consider the issues that underlie creation 
of maps and related diagrams that work in a group work context. This 
context requires representations that depict not only geospatial informa- 
tion but also individual perspectives on that information, the process 
of negotiation among those perspectives, and the behaviors (work) of 
individuals participating in that negotiation. 
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Invited Talk: 

Jon Barwise: A Heterogeneous Appreciation 



Keith Stenning 

Human Communication Research Centre, Edinburgh University 
2, Buccleuch Place, Edinburgh EH8 9LW, Scotland 
keithOcogsci .ed.ac.uk 
www .here . ed. ac . uk/Site/STENNIKE.html 



Abstract. Jon Barwise was unique amongst logicians in leading engage- 
ment between logic and other disciplines, notably linguistics, computer 
science, and the several disciplines concerned with diagrams. My main 
contact with Jon was through working on cognitive analyses of the learn- 
ing processes of students being taught logic using Hyperproof, the het- 
erogeneous environment he and John Etchemendy designed. This talk 
will trace some of Jon’s enthusiasms for interdisciplinary interactions in 
this area. I hope the audience will contribute as much or more than the 
speaker. 
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Tutorial 1: 

Formal Approaches to Visual Language 
Specification and Understanding 



Kim Marriott 

School of Computer Science and Software Engineering 
Monash University, Clayton Vic. 3168, Australia 

marr iottOcsse . monash . edu . au 
www. cs .monash. edu. au/~marriott/ 



Abstract. Two of the most fundamental questions in visual language 
research are how to specify a visual language and how to recognize and 
understand diagrams in a particular visual language. In this tutorial we 
survey the many formalisms which have been suggested over the last 
three decades for visual language specification, discuss computational 
approaches to diagram understanding based on these formalisms and 
indicate possible applications. We shall also review recent directions in 
visual language theory, notably efforts to develop an analogue of the 
Chomsky hierarchy for visual languages, the specification of diagram- 
matic reasoning, and cognitive models of visual language understanding. 
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Tutorial 2a: 

Cognitive History of Science: The Roles of 
Diagrammatic Representations in Discovery and 
Modeling Discovery 



David Gooding 



Science Studies Centre, Department of Psychology 
University of Bath, Bath BA2 7AY, UK 
hssdcgObath .ac.uk 
www.bath.ac . uk/~hssdcg/home . html 



Abstract. This session looks at some uses of diagrams in scientific 
discovery, particularly their role as intermediate representations which 
mediate between phenomena, descriptions which can be communicated 
and descriptions which are general. A range of examples will illustrate 
a variety of uses, including: the abstractive, generative role; diagrams 
as encoded knowledge; reasoning with diagrammatic representations in 
discovery; and communication (exposition and argumentation). The tu- 
torial will encourage consideration of two issues: (a) whether, from a 
cognitive standpoint, diagrams are essential to reasoning about natural 
phenomena and processes, and (b) the relationship of diagrammatic rea- 
soning to other types of visualisation and visual thinking in the sciences, 
including cognitive and computational modeling of discovery. 
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Tutorial 2b: 

Cognitive (Production System) Modelling of 
How an Expert Uses a Cartesian Graph 



Hermi Schijf 

Dept, of Educational Sciences, Utrecht University 
Heidelberglaan 2, 3584 CS De Uithof - Utrecht, The Netherlands 
h. schijf Of ss .uu.nl 



Abstract. This tutorial covers, in brief, the road from observing be- 
havior to the implementation of the observed behavior in a mixed rule- 
based and parallel network computer model. The emphasis will be on 
production system, or rule-based modelling. Rule-based modeling uses 
independently firing if-then rules to capture behavior. Why do this type 
of modeling, what types of data do you need, what are some advantages 
and limitations of the method? Simple examples of rule-based modeling 
will be given; these will be extended to a brief explanation of the mixed 
model. This model gives a theoretical explanation of the behavior of an 
expert using visual reasoning based on a Cartesian graph combined with 
verbal reasoning to teach Economics principles to students. 
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Tutorial 2c: 

The Coordination of External Representations 
and Internal Mental Representations in 
Display-Based Cognitive Tasks 



Jiajie Zhang 



Department of Health Informatics, University of Texas at Houston 
7000 Fannin Street, Suite 600, Houston, TX 77030 USA 
Jiajie . Zhang@uth.tmc . edu 
acad88 . sahs . uth . tmc . edu/ 



Abstract. Many cognitive tasks, whether in everyday cognition, scien- 
tific practice, or professional life, are distributed cognitive tasks-tasks 
that require integrative, interactive, and dynamical processing of infor- 
mation retrieved from internal representations and that perceived from 
external representations through the interplay between perception and 
cognition. The representational effect is the ubiquitous phenomenon that 
different representations of a common structure can generate dramati- 
cally different representational efficiencies, task complexities, and behav- 
ioral outcomes. A framework of distributed representations is proposed 
to account for the representational effect in distributed cognitive tasks. 
This framework considers internal and external representations as two in- 
dispensable components of a single system and suggests that the relative 
distribution of information across internal and external representations 
is the major factor of the representational effect in distributed cogni- 
tive tasks. A representational determinism is also proposed-the form of 
a representation determines what information can be perceived, what 
processes can be activated, and what structures can be learned and dis- 
covered from the specific representation. Applications of the framework 
of distributed representations will be described for three domains: prob- 
lem solving, relational information displays, and numberation systems. 
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Positive Semantics of Projections in Venn-Euler 

Diagrams 



Joseph (Yossi) Gil 1 *’**, John Howse 2 ***, and Elena Tulchinsky 1 ** 

1 Department of Computer Science 
Technion-Israel Institute of Technology 
Technion City, Haifa 32000, Israel 
2 School of Computing and Mathematical Sciences 
University of Brighton, UK 



Abstract. Venn diagrams and Euler circles have long been used as a 
means of expressing relationships among sets using visual metaphors such 
as “disjointness” and “containment” of topological contours. Although 
the notation is effective in delivering a clear visual modeling of set the- 
oretical relationships, it does not scale well. In this work we study “pro- 
jection contours” , a new means for presenting sets intersections, which is 
designed to reduce the clutter in such diagrams. Informally, a projected 
contour is a contour which describes a set of elements limited to a cer- 
tain context. The challenge in introducing this notation is in producing 
precise and consistent semantics for the general case, including a dia- 
gram comprising several, possibly interacting, projections, which might 
even be of the same base set. The semantics investigated here assigns 
a “positive” meaning to a projection, i.e., based on the list of contours 
with which it interacts, where contours disjoint to it do not change its se- 
mantics. This semantics is produced by a novel Gaussian-like elimination 
process for solving set equations. In dealing with multiple projections of 
the same base set, we introduce yet another extension to Venn-Euler 
diagrams in which the same set can be described by multiple contours. 



1 Introduction 

Diagrammatic notations involving circles and other closed curves, which we will 
call contours, have been used to representate classical syllogisms since the Middle 
Ages [7]. In the middle of the 18 t/l century, the Swiss mathematician Leonhard 
Euler introduced the notation we now call Euler circles (or Euler diagrams) [1] 
to illustrate relationships between sets. This notation uses the topological prop- 
erties of enclosure, exclusion and partial overlap to represent the set-theoretic 

* Work done in part during a sabbatical stay at the IBM T. J. Watson Research 
Center 

** Research was supported by generous funding from the Bar-Nir Bergreen Software 
Technology Center of Excellence - the Software Technology Laboratory (STL), at 
the department of computer science, the Technion 
* * * Research was supported by the UK EPSRC grant number GR/M02606 
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notions of containment , disjointness , and intersection , respectively. Another such 
notation is Venn-diagrams, named after their inventor, the 19 th century logician 
John Venn [11]. A Venn diagram contains n contours representing n sets, where 
the contours must intersect in such a way that they divide the plane into 2 n 
“zones”, which are connected regions of the plane. For every subset of the con- 
tours, there must be a zone of the diagram, such that the contours in this subset 
intersect at exactly this zone. That is to say, the zone is contained in these 
contours and in no other contour. A shaded zone indicates that the particular 
intersection of sets it denotes is empty. 

Venn diagrams are expressive as a visual notation for depicting constraints 
on sets and their relationships with other sets, but difficult to draw. 





(a) A simple and symmetrical Venn (b) The simple symmetrical Venn di- 
diagram of four contours agram of five contours 

Fig. 1.1: Venn diagrams of four and five contours. 



Fig. 1.1(a) shows a symmetrical Venn diagram of four contours, while 
Fig. 1.1(b) is the only simple symmetric Venn diagram of five contours. 

Examining these two figures, it is clear why it is so rare to see Venn diagrams 
of four or more contours used in visual formalism. Even in these simple and 
symmetrical diagrams, most regions take some pondering before it is clear which 
combination of contours they represent. 

On the other hand, Euler circles are intuitive and much easier to draw, but 
are not as expressive as Venn diagrams for two reasons. First, restricting the 
shape of contours to ovals does not allow all possible combinations of sets to be 
drawn. Second, the lack of provisions for shading restricts expressiveness even 
further. For example, there is no Euler diagram equivalent to the Venn diagram 
depicted in Fig. 1.2. 

It is therefore the case that an informal hybrid of the two notations is often 
used for teaching purposes. We use the term Venn-Euler diagrams for the nota- 
tion obtained by a relaxation of the demand that all contours in Venn-diagrams 
must intersect or conversely, by introducing shading and non-oval contours into 
Euler diagrams. Gil, Howse and Kent [ ] provided formalism for Venn-Euler 
diagrams as part of the more general spider diagrams notation. 

This paper investigates an extension to the Venn-Euler diagrams notation 
designed for the purpose of showing a set in a certain context. Intersection can 
be used for just this purpose: an intersection of A and B shows the set A in the 
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context of B and vice-versa. A projection is a contour, which is used to denote an 
intersection of a set with a “context”. 1 In Fig. 1.3(a) for example, the set Women 
is projected into the set of employees. The set Employees is called the context 
for the projection of Women. The corresponding traditional Venn-Euler diagram 
of Fig. 1.3(a) is shown in Fig. 1.3(b). 




(a) using projections (b) without projections 



Fig. 1.3: Denoting the set of all women employees. 




The projected contour (the dashed inner oval) rep- 
resents the set of women employees; it doesn’t say that 
all women are employees (as implied by Fig. 1.4). It is 
possible in this particular setting to draw a “dual” dia- 
gram, in which (taking a somewhat feminist approach) 
the set of employees could be projected into the set of 
all women. However, in general the context may com- 
prise several sets, and such a dual representation does 
not seem trivial. 

By convention, dashed iconic representation is used to distinguish projections 
from other contours. We can see that the use of projections potentially reduces 
the number of regions, with the benefit that regions that are not the focus of 
attention are not shown, resulting in less cluttered diagrams. 

Several intriguing questions arise in trying to extend the notion of projections 
beyond simple examples. 



Fig. 1.4: A Venn-Euler 
diagram stating that all 
women are employees. 



Context The notation must have a well-defined semantics 2 when a projection 
partially overlaps with a contour, and not only when it is disjoint to it or 
contained in it. Moreover, there are cases in which a projection is covered 
by a set of contours, rather than a single one. The question here is: What is 
the desired intuitive meaning of the context of a projection and how should 
this be computed from the topology of a diagram? 

Interacting Projections A diagram may contain more than one projection 
that may intersect with each other, share part of a context, or interact in 
other subtle ways. Can a projection define a context for another projection? 

1 Our use of the term projections should not be confused with the projection operator 
of relational algebra. 

2 We view diagrams as formulae of first order predicate logic. Therefore, the term 
“semantics” is used here in the mathematical logic sense, i.e., the interpretation of 
a certain formula in the model of its dictionary. 
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More generally, what is the semantics of a general topological relationship 
among several projections? 

Multi-Projections Generalizing even further, the same base set might be pro- 
jected several times into the same diagram and these projections, nicknamed 
multi-projections , might interact. We now can ask whether multi-projections 
should be allowed to interact, and what is the semantics of such an interac- 
tion? 

The projection concept was first suggested and used as part of the constraint 
diagram language [2]. However, the questions raised above complicating matters 
were not dealt with. Instead, there was a tacit understanding that only “simple” 
use of projections, which avoided these problems, was permitted. The work re- 
ported here represents an attempt to systematically deal with the semantics of 
projections. Another such attempt that took another direction, which might be 
thought of as negative or non-monotone semantics is described in [4]. The dif- 
ferences between the two approaches will be highlighted below. While selecting 
an approach might be a matter of personal preference, it is interesting to note 
how they lead to very different mathematical supporting theories. 

Outline The remainder of this paper is structured as follows. The next Sec. 2 sets 
out notation, terminology and conventions. Sec. 3 makes the case of the projection 
notation. 

Sec. 4 constitutes an informal exploration of the first issue, trying to establish an 
intuition for the notion of context. Based on this, the concept of context is precisely 
defined in Sec. 5. This makes it possible to give a precise semantics to diagrams with 
a single projection, or, more generally, diagrams in which no projection takes part in 
the context of another. 

In Sec. 6 we consider interacting projections. It is shown there that the semantics of 
these can be computed by solving a system of set equations, using what can be thought 
of as a Gaussian elimination process applied to set theory. We believe that this process 
might be of independent interest. 

Sec. 7 deals with multi-projections issue. It reaches the conclusion that it is better 
to treat multi-projections of the same base set as a single entity. Finally, Sec. 8 discusses 
related work and draws the lines of future research. 



2 Preliminaries 

To simplify mathematical formulae involving complex relationships between sets, 
we will use the arithmetical summation symbol (+) to denote set union, and 
arithmetical product (denoted by catenation) to denote set intersection. This 
notation is consistent with Boolean algebra, and will make it easier to apply the 
commutative, distributive and associative rules. We will also use, with all due 
caution, the arithmetical minus symbol (— ) to denote set minus operation. 

Even though we will ascribe precise meaning to all terms, it is convenient to 
make an initial agreement about the following, somewhat informal, terminology 
which is common in hand- waving referral to Venn and Euler diagrams. A contour 
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is a simple closed plane curve which denotes a set. A boundary contour is a 
contour that contains all other contours. A district is the set of points in the 
plane enclosed by a contour, while a zone is a region of the plane which is not 
further divided by the contours. 

For example, the Venn-Euler diagram in Fig. 2.1 
has four non-boundary contours A, B , (7, D and the 
boundary is omitted. Its interpretation includes D C 
(C-B)-A and ABC = 0. 

Mathematical preciseness dictates that in refer- 
Fig. 2.1: A Venn-Euler di- ring to a diagram, a distinction should be made be- 
agram. tween a contour and the set which it represents. For 

example, in Fig. 2.2 (which is a redraw of Fig. 1.3(a) using abstract names) the 
outer most contour is not the same as the set B which it depicts. (Ultimate 
preciseness might even distinguish between the label “B” and the set B .) 

To streamline the discourse we forgo this precise- 
ness and use B to refer also to the contour. In cases 
where the intended meaning is not obvious from con- 
text terms such as “set B" and “contour B” will be 
used. 

This convention works since the mapping from or- Fig. 2.2: A simple projec- 
dinary contours to sets is one-to-one. To deal with tion. 
projections a small extension must be made. The inner oval in the diagram 
contour represents the projection of the set D. Let D denote the set D after 
projection. We have D = DB. 

In Fig. 2.3 the base set D is projected 
twice. Since the label D is non-unique, con- 
tour labels f and g are used to distinguish 
between the two dashed contours labeled D. 

We insist that unprojected set names are up- Fig- 2.3: Projection of the same 
per case (mostly drawn from the beginning of ^ ase set i nto two different contexts, 
the Latin alphabet), while contour labels are lower case. With this convention, 
set D confined to A (resp. B) is denoted by D e (resp. D?). 

3 The Case for Projections 






Consider Fig. 3.1(a) which was constructed using More’s algorithm [8] for six 
contours. It is hard to refrain from appreciating the complexity of the drawing 
or to imagine how it can contribute to clearer visualization of set relationships. 

In sharp contrast stands Fig. 3.1(b) which shows the same 64 zones 
as Fig. 3.1(a), but in a much clearer fashion. (For technical reasons, which will 
become clear in Sec. 7, we needed to add a pseudo-boundary contour Q.) It is 
also quite easy to generalize the diagram to nine contours, by drawing a clover 
of three projections of three other sets in each of these 64 zones. Clearly, the 
process can be carried out indefinitely. 
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Fig. 3.1: A Venn diagram with six contours. 



Fig. 3.1(b) demonstrates the natural need for interactions among projections 
and for multiple projections of the same base set. 

Even Venn-Euler diagrams can get very cluttered when many contours are 
involved. The issue of clutter becomes even more crucial when such diagrams 
are used as a foundation for other, more advanced visual formalism. There are 
for example extensions of Venn-Euler diagrams in which elements of sets can 
be shown diagrammatically; these include Peirce diagrams [9,10,6] and spider 
diagrams [3]. Yet another example is the constraint diagrams notation [2] which 
uses arrows and other diagrammatic elements to model constraints not only on 
simple sets, but also on mathematical relations. 

In order to reduce clutter, and to focus attention within the diagram appro- 
priately, constraint diagrams have already made use of the projection notation. 
Consider for example the constraint diagram in Fig. 3.2(a). 




(a) with projection 




(b) without projections 



Fig. 3.2: A constraint diagrams. 



This diagram states (among other things) that the sets Kings and Queens are 
disjoint, that the set Kings has an element named Henry VIII, that all women 
that Henry VIII married were queens and that there was at least one queen he 
married who was executed. The dashed contour is a projection of the set Executed; 
it is the set of all executed people projected into the set of people married to 
Henry VIII, that is, it gives all the queens who were married to Henry VIII and 
executed. 

In the example, the inner most ellipse labeled “Executed” denotes the inter- 
section of the set Executed with the set of women who were married to Henry VIII. 
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The notation is intuitive and more concise than the alternative, which is drawing 
a large ellipse that intersects the Queens contour. As shown in Fig. 3.2(b), this el- 
lipse must also intersect with the Kings contour, or otherwise the diagram would 
imply that no kings were ever executed. To substantiate our claim that Fig. 3.2(a) 
is simpler than Fig. 3.2(b) we note that the four contours of Fig. 3.2(b) divide 
the plane into 8 disjoint areas. In contrast, the same four contours in Fig. 3.2(a) 
partition the plane into 5 such areas, which is the minimal possible number of 
non-overlapping zones. 

Moreover, Fig. 3.2(a) does not specify 
whether Henry VIII was executed or not. 
Eliminating the projections from the fig- 
ure requires delving into a history book 
and explicitly specifying this point as 
shown in Fig. 3.2(b). Alternatively, one 
could use what is known as a spider to 
refrain from stating whether or not Henry 
VIII was executed. As shown in Fig. 3.3, 
this alternative is even more cumbersome, and will probably draw the attention 
of the reader to an irrelevant point. 







Fig. 3.3: Using a spider notation to 
preserve the semantics of Fig. 3.2(a) 
while eliminating projections from it. 



4 Intuitive Semantics of Projections 

Taking a rather informal and intuitive approach this section explores the notion 
of the context of a projection, setting the guidelines for the precise definitions 
in the following section. 




(a) A projection into (b) A projection into (c) A projection with 

a context defined by multiple contexts two minimal contexts 

multiple contours 

Fig. 4.1: Variations in the context of a projection. 




Perhaps the first thing to notice is that there are cases in which a projection 
is not into a single contour but rather into an area defined by multiple contours. 
This situation is depicted in Fig. 4.1(a), from which we surmise D = D(B + C ). 
This conclusion follows from the simple observation that the inner oval is not 
contained in the one labeled B , nor the one labeled C. Hence none of these two 
contours can define by itself a context for the projection of D. 

Consider now Fig. 4.1(b). An awkward way to interpret the diagram that 
would still be consistent with our interpretation of Fig. 2.2 is D = DA C B. 
This interpretation seems quite contrived because it does not select a minimal 
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“context” for D , and stands in disagreement with our traditional mathematical 
and logical thinking. In trying to delineate the boundaries of any concept, or 
in making a definition, one strives to make the strongest possible constraint. 
The statement “platypusii live in Australia” is more informative and useful than 
“platypusii live on planet earth” . Our semantics for the context notion follows 
this line of thought. 

Conversely, suppose that we allow a non-minimal context to be chosen in 
defining a context for a projection. Then, how should a selection among many 
possible contexts be made in a consistent manner? If, for example, a “maximal” 
context is to be selected in all cases, then even Fig. 1.3(a) loses its natural 
semantics, since the maximal context is the boundary contour. Projection where 
the boundary serves as a context is nonsensical, since the boundary is an ominous 
set which is supposed to contain the whole universe. Thus, a projection of the 
sort depicted in Fig. 4.2 is deemed illegal. 

The alternative, and much more intuitive, se- 
D, — mantics of Fig. 4.1(b) is simply D = DB with 

') the additional requirement B C A. We have thus 
discovered that if there are multiple contexts for 
Fig. 4.2: An illegal projection. a projection only the minimal one should be con- 
sidered. The understanding of a projection should 
not change if we add a contour that contains its context. However, as demon- 
strated in Fig. 4.1(c), (which can be thought of as a dual to Fig. 4.1(b)) there 
could be cases in which a contour has multiple minimal contexts. 





(a) A projection and 
a contour in different 
contexts 




(b) A projection into 
multiple, non-simple 
contexts 




(c) A projection dis- 
joint to a contour in 
the same context 



Fig. 4.3: More variations in the context of a projection. 



Since D is contained in B , we would like to maintain that D = DB. This 
however contradicts the symmetrical demand that D = DC. From the natural 
language demand that “D is the set of D ' s which are in the context of B , and in 
the context of C” , we obtain that D = D(BC ), which is in agreement with the 
topology of the diagram, since the contour D is constrained to the zone defined 
by the intersection of B and C. 

Fig. 4.1(a) gave an example in which several contours together provided a 
context for a projection. The minimality of the context is not only with respect 
to containment, but also with regard to the contours taking part in the context. 
For example, in Fig. 4.3(a) only B is considered a context of D even though 
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all of {B,D,E}, {B,E}, {B,D}, and {E,D} cover B. Similarly, we are not 
interested in a context of a contour which comprises of the contour itself. 

Fig. 4.3(b) may at first seem complicated. A moments reflection will reveal 
that it is nothing more than a generalization of Fig. 4.1(a) and Fig. 4.1(c). 
Indeed, the projection is contained in the union of B\ and C\ as well as the 
union of B 2 and C 2 . Thus, D = D {{B\ + C\){B 2 + C 2 )). 

The semantics of projections was so far defined by what may be called a 
contour-based, approach , which computes the projection with respect to all of 
its “minimal contexts” , where a context is defined as a union of a collection of 
contours. An alternative definition follows from what might be called a zone- 
based approach. In this approach, the semantics of a projection is defined in 
terms of the zones in which it falls, and where each of these zones is appropriately 
defined by the set of contours which contain it, as well as the set of contours 
disjoint to it. 

In some of the preceding examples, both approaches lead to the same se- 
mantics. For example, in Fig. 4.1(c), set D is projected into the zone BC , while 
in Fig. 4.1(a) it is projected into the union of three zones: BC, B — C , and C — B 
and 



D = D {{BC) + {B-C) + {C- B)) 

= D{B + C ). 

A diagram which clearly distinguishes between the two approaches is depicted 
in Fig. 4.3(c). In the zone-based approach we have D = D{A — E), while in the 
contour-based approach D = DA, with the additional statement that DE = 0. 
We see that in the zone-based approach the semantics of contour is defined not 
only in terms of the contours in which it is contained, but also in terms of the 
contours which are disjoint to it. Conversely, in the contours-based approach uses 
“positive semantics”, in which only the contours that intersect with a projection 
contribute to its semantics. The zone-based approach was investigated in [4] . In 
this paper, we concentrate on the contours-based approach. 



5 Covers and Contexts 

Through a series of definitions, this section develops a formal notion of a con- 
text, k{c ), of a contour c (Def. 6). The territory of c, denoted by r(c) (Def. 5) is 
the entire set of contours which take part in the definition of a context. 

Although all definitions carry a geometrical and topological intuition, for the 
sake of preciseness we rely on a pure, set theoretical mathematical model. In this 
model, a (Venn-Euler) diagram is nothing but a set of contours and zones. 

Definition 1. A diagram is a pair { C,Z ) of a finite set C of objects, which we 
will call contours, and a set Z of non-empty subsets of C, which we will call 
zones, such that Vc £ C,3z £ Z,c £ z. 
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The most simple diagram is an empty one: (0,0). A more interesting example 
is the pair 

({A, B, C, D , E} , { z i, z 2 , z 3 , z 4 , z 5 , z 6 , z 7 , z s , z 9 }), (1) 

where 

zi = {A} z 2 = {A, Bj z 3 = {A, C} 

z 4 = {A, B, D} z 5 = {A, C, D} z 6 = {A, B, C, D} (2) 

z 7 = {A,B,C} z 8 = {A,E) zq = {E} 

is a diagram since A £ z\, B £ z 2 , C £ z 3 , D £ z 4 , and E £ zg. This diagram 

will serve as our running example for the remainder of this section. 

The geometrical interpretation of 
the Def. 1 is that each zone is thought of as 
the set of contours in which it is contained. 
It is implicitly disjoint to the remaining con- 
tours. The requirement Vc £ C,3z £ Z,c £ z 
ensures that there are no degenerate con- 
tours. Def. 1 does not include a boundary 
contour. Accordingly, we do not consider 
the zone which is outside all contours, and 
therefore zones must be non-empty sets of contours. 

In many cases it is possible to obtain a layout of a diagram from this inter- 
pretation. One possible layout of our running example is given in Fig. 5.1. Note 
that in this particular layout zone z 7 is mapped to a noncontiguous area of the 
diagram. However, Fig. 5.1 is useful for gaining insight into our definitions. 

A model for a diagram is a set U, called the universe and a semantic func- 
tion if : C — » 2 W , mapping each contour to a subset of the universe. Let us extend 
the domain of if to include Z as follows. For all z £ Z, 

if(z) = \\_ip(c)- ^ ip{c). (3) 

cE-z cEC— 2 : 

In words, the semantics of z is the intersection of the semantics of the contours 
containing it, minus the semantics of all contours which do not contain it. 

The semantics of the diagram is given in 

^iP(c) = ^2 i/j(z). (4) 

cec zez 

Equation (4) is called in the literature [3] the plane tiling condition, due to the 
geometrical intuition behind it. In words, this condition says that the list of 
zones is comprehensive i.e. , every set member, must be a member of one of its 
zones. 

Definition 2. The district of a contour c is d(c) = {z £ Z\c £ z}. The 
district of a set of contours S is the union of the districts of its contours 
d (S) = E ce s d ( c )- 




Fig. 5.1: A diagram with five con- 
tours and nine zones. 
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A district is basically a collection of zones. 

In our example, we have that d(A) = {z 7 , z 2 , z 3 , z 4 , 25 , zq, z 7 , z 3 }- Also, the 
district of the set { B , C} is the union of districts of its respective elements: 

d({B, C} = {z 2 , z 4 , z 6 , z 7 } + {z 3 , z 5 , z 6 , z 7 } 

= {Z2,Z 3 ,Z4,Z 5 ,Z 6 ,Z 7 } . 

Although our objective is to develop a notion of context for a single contour, 
it is mathematically easier to do so for the more general case, which is a set of 
contours. The overloading of the term district in Def. 2 makes it possible to treat 
both contours and sets of contours in a similar fashion, avoiding the cumbersome 
equating of a contour c with the singleton set {c}. For the remainder of this 
section, we let the variables X, Y and Z range over C + 2 C . 

Using the district notion, we can define a partial order on C. 

Definition 3. We say that X is covered by Y if d(X) C d(Y). We say that X 
is strictly covered by Y if the set containment in the above is strict. 

From a topological standpoint, a contour (or a set of contours) is (strictly) 
covered by another contour (or a set of contours), if the area defined by the 
first is (strictly) contained by the area defined by the second. In the above 
example, the contour A strictly covers contour D as well as the set {B,C,D}. 
The set {B, C, E} strictly covers the set {D, E}, as well as the single contour B. 

Some covering relationships are less “interesting” than others. Let S , S' C 2 C . 
Then, if S covers X, then the set S + S' is (trivially) a cover of X. Stated 
differently, there might be some members of S which do not “contribute” to its 
coverage of X. In an extreme case, S could be a strict cover of X. But, when 
we remove from S those members of it whose district is disjoint to that of X, S' 
ceases to be a strict cover of X (although it remains a cover of it) . The following 
definition tries to capture a more meaningful notion of coverage. 

Definition 4. A set of contours S is a reduced cover of X if S strictly cov- 
ers X, XS = 0 , and there is no S' G S such that S' covers X. 

The natural extension of the above definition is that if a single contour is a 
strict cover of X then it is also a reduced cover of X. In our example, A (just 
as {A}) is a reduced cover of D and of {B, C}. 

Lemma 1 . Let S , a set of contours, be a reduced cover of X . Then, for all c G S, 
d(c)d(X) yf 0 . (In words, no member of S can be disjoint to X.) 3 

In a sense, a reduced cover of X gives a context of X. The collection of these 
is called territory. 

Definition 5. The territory of X is r(X) = {S C C\S is a reduced cover of X}. 
The proof of this, and all other subsequent claims, is omitted for space reasons. 
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In our running example, we have that t(D) = {{A} , {B, C}}. We are now in 
a good position to define the context of a contour or of a collection of contours. 

Definition 6. The context of X, r(X) ^ 0 is defined as 



If on the other hand t{X) = 0, we say that X is context free. 

The context of a contour (or a set of contours), when it exists, is a set of 
zones. In our example, the context of D is 

k(D) = d(A)d({B , C}) = d(A)(d(B) + d(C )) = {z 2 , z 3 , z 4 , z 5 , z 6 , z 7 } . 

Similarly, we have k({B, C}) = d(A) = {zi, z 2 , z 3 , z 4 , z 5 , z 6 , z 7 , z 8 }. 

Now it should be clear why we used the set of all reduced covers, rather than 
simply the set of all strict covers in the definition of territory. If we were to 
compute the intersection of all districts of sets which strictly cover E, then this 
intersection would have been also over { E , B} and {E, C}. Clearly, the result of 
such an operation would have been d(E ), which is hardly useful for defining a 
context for E. In our example, both A and E are context-free. 

6 Interacting Projections 



Fig. 6.1: Speakers of English this context. In this section we formalize this 

among Hebrew speakers among process, while dealing with the more general 
Israeli Citizens. case, in which the context of a projection may 

contain other projections as well. In Fig. 6.1 for example, we have that the set of 
English speakers, denoted E, is projected into the set formed by projecting the 
set of Hebrew speakers (denoted H) into the set of Israeli citizens (denoted I). 

These conditions can be summarized in the following set of equations: 



A more challenging example is given in Fig. 6.2 in which the context of each 
of the two projections is defined by the other projection. The figure gives rise to 
a system of equations 



k(x)= n d(s)= n 



ser(x) serpOces 




The previous section solved the problem 
of computing the semantics of a diagram with 
only one projection in it. Rather informally, 
we say that the semantics of the projection 
is the intersection between the base set and 



H = HI 

E= EH = EHI. 



H = H{I+E) 
E = E(U + H) 



( 5 ) 

( 6 ) 
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where U denotes the set of citizens of the UK. Substituting (6) into (5) gives a 
single equation with a single unknown H: 

H = H(I + E{U + H)) 

= HI + HEU + HEH (7) 

= aH + ft, 

where the coefficients 



a = HE 

ft = HI+ HEU = H(I + EU) 


(8) 


depend only on the given sets H, /, E, and U. 

The following lemma is pertinent in solving (7). 

Lemma 2. Let a and ft be two given sets. Then , the equation 




x = ax + ft 


(9) 


holds if and only if 




ft C x C a + ft. 


(10) 


Applying Lemma 2 we see that (7) has multiple solutions, 
only the minimal one, namely 


We argue that 


H = ft = H(I + EU ) 


(11) 



makes sense in assigning semantics H. On 
the one hand, (11) takes the format of pro- 
jecting H into the context I + EU, i.e., the 
set of Hebrew speakers among either Israeli 
citizens or the English speaking citizens of 
the UK. This is in agreement with the topol- 
ogy of the contour denoted H in Fig. 6.2. On 
the other hand, a non-minimal solution to (7) 
must contain an element j, such that 

je(a- P) (12) 

It easily follows from (12) and the definitions of a and ft (8) that j ^ (/+[/). In 
words, j is neither a citizen of Israel nor of the UK. This stands in contrast to 
the fact that the contours denoted I and E in Fig. 6.2 cover both projections. 

The generalization of the path we just pursued is largely technical. 

Definition 7. A projections diagram is a diagram (C,Z), with some set V C C 
of contours which are marked as projections. A projections diagram is legal only 
if all of its projections have a context (Def. 6). 




Fig. 6.2: Speakers of English and 
Hebrew among Israeli and UK Cit- 
izens. 
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To give semantics to a projections diagram, we use a labeling function x '■ 
C -> 2 U which maps each of the contours in a projections diagram to some 
subset of the universe. For c £ C — V, the semantics function is defined simply 
as ip(c) = x(c). The problem is to associate a projected semantics, in the form of 
the function ip(c), with all c£V. (For all such c, x(c) is the set being projected, 
and if(c) is its projected meaning.) 

After ij}(c) is computed for all c, we can 
extend its domain to zones as before (3), 
and then use the plane tiling condition (4) 
to write the semantics of the diagram. Start- 
ing for example from Fig. 6.2, then after solv- 
ing for for all projected contours we can 
proceed with the non-projection diagram of the semantics of the projected con- 
Fig. 6.3. tours was computed. 

It follows from the leading discussion that the projected semantics of a con- 
tour p £ V is given by 

i>(p)=x(p) n ( 13 ) 

5Gr(p) c£5 

Examining (13) more closely, we see that it is in fact a set of polynomial 
(so to speak) equations over subsets of U. The unknowns in this system are 
the values of 4’{p) for p € V. For brevity, let us denote these as Xi , . . . ,x n . The 
constants in this system are the values of x(c), where c £ C, which we will denote 
as cti, . . . cr m . The structure of the equations is determined by the “topology” of 
the diagram, i.e., the relationships between Z and C, as expressed in the contexts 
of each of the projections contours. We can rewrite (13) as 

xi Pi (rr^ , . . . , c 7 m , X 2 3 ■ • • j Xn) 

: (14) 

X n — P n (<7l , , (7 rn , X \ , . . . , X n — 1 ) 

where Pi, i = 1 , . . . ,n, is a multivariate positive set polynomial (i.e., a formal 
expression involving only set union and intersection) over < 7 i , . . . , a m and 

X\ , . . . , Xi— 1 , Xi^-i , . . . X n . 

The following lemma is pertinent in deriving a general process of solving (14). 

Lemma 3. Every multivariate positive set polynomial P over variables 
di,...,St,x can be rewritten as P(5i, . . . , Se, x) = Pi(Si, . . . , Sfjx + 
P 2 (5i,.. . ,Se). 

In the case n = 2, a solution for (14) is obtained by substituting the equation 
defining x 2 in the equation defining x\. By applying Lemma 3 we have that the 
result of the substitution is a linear equation in x\ . We then proceed to finding a 
minimal solution to this equation using Lemma 2. To solve for x 2 we substitute 
into the equation defining it, the solution for x\. 




Fig. 6.3: A redraw of Fig. 6.2 after 
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The procedure is only slightly more complicated for n > 2. In fact, it can 
be thought of as a Gaussian elimination process applied to set equations. Us- 
ing Lemma 3 we rewrite the equation x n = P n (<Ti,... ,cr m ,xi, . . . ,x n -i) as 
a linear equation in x n in the format of Lemma 2. We solve for x n in terms 
of <Ti, . . . , <T m and x \, . . . , x n -\ 1 using Lemma 2 and choosing as usual the min- 
imal solution. We then substitute this solution into Pi, . . . ,P n - 1 , whereby re- 
ducing the number of variables by one. The process is then iterated until there is 
only one equation with only one unknown x\. We solved this and express X\ in 
terms of the constants ay, . . . , <j m . In backtracking these steps, with appropriate 
substitutions of the values of the unknowns which were already we can write the 
minimal solution for £ 2 , ■ • ■ , x n in order in terms of the constants only. 

7 Multi-Projections 




(a) Intersecting multi-projections (b) Disjoint multi-projections into in- 
into intersecting contexts tersecting contexts 

Fig. 7.1: Multi-projections into intersecting contexts. 

So far, the discussion assumed that every base set is projected only once 
into a diagram. However, an additional expressive power of projections arises 
by introducing multi-projections , i.e., the ability to make several projections of 
the same base set into a diagram. Multi-projections enable the drawing of Venn 
diagrams with a large number of contours in a structured, visually-clear fashion, 
as demonstrated in the Venn diagrams of six contours Fig. 3.1(b). 

There are some simple cases in which the desired semantics of multi- 
projections is clear. In Fig. 7.1(a) for example we have 



D f = DB 



D 9 = DC. 



(15) 



The same equations (15) hold also for Fig. 7.1(b). An additional constraint in 
this diagram is that, since contour / is disjoint to C in this diagram, D^C = 0, 
i.e., 



DBC = 0. (16) 

Equation (16) follows also from the fact that g is disjoint to B, and also from / 
being disjoint to g. 
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Obtaining a semantics for simple examples such as Fig. 7.1(a) and Fig. 7.1(b) 
is easy to do with only a slight generalization to the semantics as described 
in the previous section. We can try to define a multi-projection diagram as a 
projection diagram, adorned with an equivalence relationship A C V x V . With 
this definition we will say that contours p\ and P 2 are multi-projections , i.e., 
projections of the same base set if and only if (p\,p 2 ) £ A. The labeling function 
must then satisfy 

(Pi,P2) S A => x(Pi) = X(P2)- (17) 

Under this requirement, the system of equa- 
tions (14) is generated as usual and the semantics 
is computed by using the Gaussian elimination 
process as described above. 

Unfortunately, this simple-minded approach 
leads to surprises in many cases. In Fig. 7.2 for 
example, 




Fig. 7.2: Multi-projection into 
contained contexts. 



D f = DB 

(18) 

D 9 = D{BC). 

However, since / and g are disjoint it follows that 
0 = D f D 9 

= DBC (19) 

= D 9 , 

even though the diagram may suggest that D 9 is not necessarily empty. 

A remedy to this, and other more intricate problems (which are not dis- 
cussed here for space constraints), might be a demand that the contexts of 
multi-projections are disjoint, but this would render illegal Fig. 3.1(b). Instead, 
we take an approach which allows not only projections to show up several times 
in a diagram, but also ordinary contours. 

Recall that a diagram was defined (Def. 1) in abstract terms, which could be 
described topologically on the plane. In laying out the running example of Sec. 5 
(the diagram defined by (1) and (2)), in Fig. 5.1, one zone ( 27 ) was mapped to a 
noncontiguous area. Let us now generalize in allowing layouts in which contours, 
and not only zones, can be mapped into noncontiguous areas, as was done in 
Fig. 7.3. 

The figure gives an alternative layout of the running example, in which the 
abstract contours E and D are each mapped into three noncontiguous areas of 
the plane. Contours E and D can therefore be thought of as “multi-contours” 
with several fragments in the diagram. Note that zones zg and zg become non- 
contiguous while z 7 is contiguous in this layout. 

As revolutionary as multi-contours may look, at this stage they have abso- 
lutely no effect on our definitions and algorithms which relied on an abstract 
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definition of a diagram, devoid of any geometrical or topological connotation. 
The main effect is on layout editors, such as CD Editor [5], which must be smart 
enough to treat noncontiguous zones and contours as one. For example, if a zone 
is shaded, then this shading should be applied to all the areas it comprises. 

The following definition promotes multi- 
contours and fragments from the intuitive- to 
the precise conceptual level. 




Fig. 7.3: A noncontiguous layout of 
Fig. 5.1. 

Definition 8. A multi-projection diagram is a diagram (Def. 1 ), adorned with 
a set PCC, of contours marked as projections, as well as a contour fragments 
function <j> : C — > 2 2 such that 

Vc <E C • *=d(c ) (20) 

Sg0(c) 

In words, the function f> maps each contour 
into its fragments, where a fragment of a contour 
is defined as a non-empty set of zones. Fragments 
are distinguished by the set of zones in them. In 
addition, it is required that the contour is “the 
sum of its fragments”, i.e., that the union of the Fig. 7.4: Redrawing Fig. 7.1(b) 
zones defined in the fragments is the same as the after computing the semantics 
zones in (the district of) the contour. If (f>(c) = °f the multi-projection. 

|d(c)} holds for a contour c, then we say, by a slight abuse of terminology, that c 
is unfragmented. If on the other hand |</>(c)| > 1 we say that c is a multi-contour 
with each of the members of </>(c) considered its fragments. 

For example, the fragments function inferred from Fig. 7.3 is given by 
= {{^ 4 } , {- 5 } , {^e}}, and </>(E) = {j> 8 } , { 29 } , {^ 8 , 29 }}, while con- 
tours A, B , and C are unfragmented. It is important to keep in mind however, 
that although it is in many cases possible to produce a layout for a multi- 
projection diagram, such a diagram remains a pure set-theoretical creature. 

With Def. 8, a multi-projection is a single projected contour which has mul- 
tiple fragments. Taking this approach, the semantics of a multi-projection is 
strikingly simple, and is basically a rehash of the procedure described in Sec. 6, 
ignoring the fragments altogether! 

1. Produce the system of equations (14), while considering every multi- 
projection as a single contour. This is to say that the context of each multi- 
projection is determined with complete disregard to its fragments. 

2. Compute the projected semantics of each multi-projection using the process 
describe above in Sec. 6. 

3. Apply the plane tiling condition, ignoring the fragments of every multi- 
projection, those of any other multi-contour. 
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In applying the above to the diagram of Fig. 7.1(b), we have that the D 
is a multi-projection contour with context B + C. Therefore the semantics is 
D = D(B + C ), which after replacement produces the multi-diagram without 
projection depicted in Fig. 7.4. 

Applying the plane tiling condition to Fig. 7.4 
we obtain that D(BC ) = 0, which is the same as 
(16), but without going through the unnecessary 
step of defining a semantics for each fragment as 
done in (15). 

It is important to notice that after the sys- 
tem of equations was solved, the extension of the 
function ip to zones makes it also possible to assign semantics to each of the 
fragments of a multi-contours as follows. For all / £ 4>{c), we extend ip to it as 

V’(Z) = Eze/V’O)- 

It is not difficult to check that the semantics of a contour is the same as the 
union of the semantics of its fragments ip(c) = E/e<Wc) V’(Z)- 

In Fig. 7.2, for example, we find D = DB , without the incurred difficulty 
in (19). In this case we have that the semantics of the fragment / is D(B — C ) 
while that of fragment g is D(BC). 

In concluding our discussion of multi-projections, it is interesting to recon- 
sider Fig. 7.1(a) in light of Def. 8. Redrawing this figure while highlighting some 
of the odd shaped zones, we see (Fig. 7.5) that zone z\ = {B, C} occupies a non- 
contiguous area of the plane. Zone Z 2 = {B,C, D} has an even more degenerate 
layout, in which the area it occupies on the plane is somewhat artificially parti- 
tioned by contour fragments. This anomaly accentuates the fact that a “better” 
layout of the diagram would be as in Fig. 4.1(a). 

This situation could have been eliminated by requiring that for every multi- 
contour c, and for every two distinct fi, f 2 £ <P(c), / 1/2 = 0. We refrained from 
making this requirement since the aesthetic end it serves is not very essential. 

8 Conclusions and Future Research 

It remains open to find an efficient algorithm for producing and then solving this 
system of equations, or proving that no such algorithm exists. It is intriguing to 
determine whether the contour-based approach, or the zone-based approach [4], 
or perhaps a third one, is found by users to be more intuitive. 

Acknowledgments. We pay tribute to Yan Sorkin, the creator of CDEditor, the au- 
tomatic tool which was used in generating virtually all diagrams presented here. The 
production of this manuscript would have impossible without his Sisyphean efforts in 
making updates to the editor as the research on semantics progressed. 




Fig. 7.5: Degenerate layout of 
zones in Fig. 7.1(a). 
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Abstract. Spider diagram systems provide a visual language that 
extends the popular and intuitive Venn diagrams and Euler circles. 
Designed to complement object-oriented modelling notations in the 
specification of large software systems they can be used to reason 
diagrammatically about sets, their cardinalities and their relationships 
with other sets. A set of reasoning rules for a spider diagram system is 
shown to be sound and complete. We discuss the extension of this result 
to diagrammatically richer notations and also consider their 
expressiveness. Finally, we show that for a rich enough system we can 
diagrammatically express the negation of any diagram. 

Keywords Diagrammatic reasoning, visual formalisms. 



1 Introduction 

Euler circles [2] is a graphical notation for representing relations between classes. 
This notation is based on the correspondence between the topological properties of 
enclosure, exclusion and intersection and the set-theoretic notions of subset, disjoint 
sets, and set intersection, respectively. Venn [14] modified this notation to illustrate 
all possible relations between classes by showing all possible intersections of contours 
and by introducing shading in a region to denote the empty set. Flowever a 
disadvantage of this system is its inability to represent existential statements. 
Peirce [11] modified and extended the Venn system by introducing notation to 
represent empty and non-empty sets and disjunctive information. Recently, full 
formal semantics and inference rules have been developed for Venn-Peirce 
diagrams [13] and Euler diagrams [6]; see also [1, 5] for related work. Shin [13] 
proves soundness and completeness results for two systems of Venn-Peirce diagrams. 

Spider diagrams [3, 7, 8, 9] emerged from work on constraint diagrams [4, 10] and 
extend the system of Venn-Peirce diagrams investigated by Shin. Constraint diagrams 
are a visual diagrammatic notation for expressing constraints such as invariants, 
preconditions and postconditions that can be used in conjunction with the Unified 
Modelling Language (UML) [12] for the development of object-oriented systems. 
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In [8] we considered a system of spider diagrams called SD2 that extended the 
diagrammatic rules and enhanced the semantics of the second Venn-Peirce system 
that Shin investigated (i.e., Venn-II, see [13] Chapter 4) to include upper and lower 
bounds for the cardinality of represented sets. In the proof of completeness of SD2 we 
opted for a strategy in which the diagram that results from combining a set of 
diagrams and the diagram that is the consequence of that set are expanded in a way 
similar to the disjunctive normal form in symbolic logic. This strategy extends to 
other similar systems, including the one considered in this paper. In this paper, we 
extend SD2 by introducing new notation and extending the inference rules to cover 
this notation. This extended system is shown to be sound and complete. 

A discussion of the system SD2 is conducted in section 2, where the main syntax 
and semantics of the notation is introduced and soundness and completeness results 
are given. In section 3 we introduce new notation into the SD2 system, extend the 
inference rules described in section 2 to include the new notation and show that the 
extended system is sound and complete. We also enrich the system by providing 
additional results for reasoning with more expressive diagrams. In section 4, we show 
that we can express diagammatically the negation of every extended SD2 diagram. 
Section 5 states the conclusions of this paper and details related, ongoing and future 
work. Throughout this paper, for space reasons, we omit most proofs. 



2 Spider Diagrams: SD2 

This section introduces the main syntax and semantics of SD2, a system of spider 
diagrams. For further details see [8], 



2,1 Syntactic Elements of Unitary SD2 Diagrams 

A contour is a simple closed plane curve. A boundary rectangle properly contains all 
other contours. A district (or basic region ) is the bounded subset of the plane enclosed 
by a contour or by the boundary rectangle. A region is defined, recursively, as 
follows: any district is a region; if r\ and r 2 are regions, then the union, intersection, or 
difference, of /q and r 2 are regions provided these are non-empty. A zone (or minimal 
region ) is a region having no other region contained within it; a zone is uniquely 
defined by the contours containing it and the contours not containing it. Contours and 
regions denote sets. 

A spider is a tree with nodes (called feet) placed in different zones; the connecting 
edges (called legs) are straight lines. A spider touches a zone if one of its feet appears 
in that region. A spider may touch a zone at most once. A spider is said to inhabit the 
region which is the union of the zones it touches. For any spider s, the habitat of s, 
denoted rj(s), is the region inhabited by s. The set of complete spiders within region r 
is denoted by S{r). The set of spiders touching region r is denoted by T(r). A spider 
denotes the existence of an element in the set denoted by the habitat of the spider. 
Two distinct spiders denote distinct elements. 
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Every region is a union of zones. A region is shaded if each of its component zones 
is shaded. A shaded region denotes the empty set if it is not touched by any spider. A 
unitary’ SD2 diagram is a single boundary rectangle together with a finite collection of 
contours (all possible intersections of contours must occur, i.e., the underlying 
diagram is a Venn diagram), spiders and shaded regions. Each contour must be 
labelled and no two contours in the same unitary diagram can have the same label. 
The labelling of spiders is optional. For any unitary diagram D, we use C = C(D), 
Z = Z(D), Z* = Z* (£>), R = R{D), R* = R*(D), L = L(D ) and S = S(D) to denote the 
sets of contours, zones, shaded zones, regions, shaded regions, contour labels and 
spiders ofD, respectively. 




The SD2 diagram D in Figure 1 can be interpreted as: 

A-(BkjC) = 0 a \<\C-(AkjB)\<2 a 2<\C — B\ a 1<|5|. 

2.2 Semantics of Unitary SD2 Diagrams 

A model for a unitary SD2 diagram D is a pair m = (U, l F) where U is a set and 
: C — > Set U, where Set U denotes the power set of U, is a function mapping 
contours to subsets of U. The boundary rectangle U is interpreted as U, V F( U) = U. 

The intuitive interpretation of a zone is the intersection of the sets denoted by 
those contours containing it and the complements of the sets denoted by those 
contours not containing it. We extend the domain of ¥ to interpret regions as subsets 
of U. First define *P : Z — > Set U by 

V(z)= P| x P(c)n P|T^) 

ceC + (z) ceC~(z) 

where C + (z) is the set of contours containing the zone z, C~(z ) is the set of contours 
not containing z and 'F(c) = U - 'F(c) , the complement of l F(c). Since any region is a 
union of zones, we may define : R^> Set U by 

¥(/-)= U'F(z) 

zeZ(r) 

where, for any region r, Z(r) is the set of zones contained in r. 

The semantics predicate Po(m) of a unitary diagram D is the conjunction of the 
following two conditions: 
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Distinct Spiders Condition: The cardinality of the set denoted by region r of unitary 
diagram D is greater than or equal to the number of complete spiders in r. 

/vjT'(r)! > |S(r)| 

reR 

Shading Condition: The cardinality of the set denoted by a shaded region r of unitary 
diagram D is less than or equal to the number of spiders touching r. 

/\J l F(r)| < |T(r)| 

reR* 



2.3 Compound Diagrams and Multi-diagrams 

Given two unitary diagrams D and D 2 , we can connect D, and D 2 with a straight 
line to produce a diagram D = D t -D 2 . If a diagram has more than one rectangle, then 
it is a compound diagram. The ‘connection operation’ is commutative, D\-D 2 = D 2 - 
D\. Hence, if a diagram has n unitary components, then these components can be 
placed in any order. 

The semantics predicate of a compound diagram D is the disjunction of the semantics 
predicates of its component unitary diagrams; the boundary rectangles of the 
component unitary diagrams are interpreted as the same set U. 




The compound diagram D = D l -D 2 in Figure 2 asserts that: 

(3 x,y xeHnC a yeB-C a \Ar\C-B\<\) v 
(3x, y • xeB a yeA-(B(jC) a |H-i?|=l). 

A spider multi-diagram is a finite collection A of spider diagrams. The semantics 
predicate of a multi-diagram is the conjunction of the semantics predicates of the 
individual diagrams; the boundary rectangles of all diagrams are interpreted as the 
same set U. Contours with the same labels in different individual diagrams of a multi- 
diagram A are interpreted as the same set. 

In [8] we describe how to compare regions across diagrams. This formalizes the 
intuitively clear notion of ‘corresponding regions’ in different diagrams. For example, 
in figure 3, the region z = z l u z 2 in D corresponds to the zone z in D' since both 
represent the set B - A. 
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D 



D' 




U 



Figure 3 



2.4 Compliance and Consistency 

A model m = (U, M') complies with diagram D if it satisfies its semantic predicate 
P D (m). We write m 1= D. That is, m \= D <^> P D {m). Similarly, a model m complies 

with multi-diagram A if it satisfies its semantic predicate PJn /). That is, m 1= A o 
PJtn). A diagram is consistent if and only if it has a compliant model. All SD2 
diagrams are consistent. However, there exist inconsistent SD2 multi - diagrams [8]. 



2.5 Rules of Transformation for SD2 

in [8], we introduced rules that allow us to obtain one unitary diagram from a 
given unitary diagram by removing, adding or modifying diagrammatic elements. 
These rules are summarised below; they are based on the rules given by Shin in [13], 
which developed earlier work of Peirce [11]. 

Rule 1: Erasure of shading. We may erase the shading in an entire zone. 

Rule 2: Erasure of a spider. We may erase a complete spider on any non-shaded 
region. 

Rule 3: Erasure of a contour. We may erase a contour. When a contour is erased: 

• any shading remaining in only a part of a zone should also be erased. 

• if a spider has feet in two regions which combine to form a single zone with the 
erasure of the contour, then these feet are replaced with a single foot connected to 
the rest of the spider. 

Rule 4: Spreading the feet of a spider. If a diagram has a spider s, then we may 
draw a node in any non-shaded zone z that does not contain a foot of s and connect it 
to s. 

Rule 5: Introduction of a contour. A new contour may be drawn interior to the 
bounding rectangle observing the partial-overlapping rule: each zone splits into two 
zones with the introduction of the new contour. Each foot of a spider is replaced with 
a connected pair of feet, one in each new zone. Shaded zones become corresponding 
shaded regions. 

Rule 6: Splitting spiders. If a unitary diagram D has a spider s whose habitat is 
formed by n zones, then we may replace D with a connection of n unitary diagrams 
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D\- ... -D„ where each foot of the spider s touches a different corresponding zone in 
each diagram D 

Rule 7: Rule of excluded middle. If a unitary diagram D has a non-shaded zone z 
where |S(z)| = n, then we may replace D with D\—D 2 , where D\ and D 2 are unitary and 
one of the corresponding zones of z is shaded with |S(z)| = n and the other is not 
shaded with |S(z)| = n+l. 

Rule 8: The rule of connecting a diagram. For a given diagram D, we may connect 
any diagram D' to D. 

Rule 9: The rule of construction. Given a diagram D x - ... -D m we may transform it 
into D if each D t ,..., D n may be transformed into D by a sequence of the first eight 
transformation rules. 



2.6 Consistency of a Multi-diagram and Combining Diagrams 

Definition: An a diagram is a diagram in which no spider’s legs appear; that is, the 
habitat of any spider is a zone. 

Any SD2 diagram D can be transformed into an a diagram by repeated application 
of rule 6, splitting spiders. 

Two unitary a diagrams with the same contour set are consistent if and only if for all 
zones 

(i) corresponding shaded zones contain the same number of spiders; 

(ii) when a shaded zone in one diagram corresponds to a non-shaded zone in the other, 
the shaded zone contains at least as many spiders as the non-shaded zone. 

Two diagrams D 1 and D 2 are consistent if they can be transformed into a diagrams 
with the same number of contours D Xa and D lh (by rules 5 and 6) and there exist 
unitary components D Xa of D la and Df h of D 2b such that D Xa and D 2h are consistent. 
See [8] for details. 

Intuitively, the diagrammatic conditions (i) and (ii) would prevent the case in 
which two corresponding zones denote two sets whose cardinalities are inconsistent; 
this is the only case in which a pair of unitary a diagrams can be inconsistent. 

Given two consistent diagrams, D l and D 2 , we can combine them to produce a 
diagram D — D l * D 2 , losing no semantic information in the process. Given D x and 
D 2 , first transform them into a diagrams D la and D lh with the same number of 
contours (using rules 5 and 6). Then the combined diagram D = D l * D 2 is the 
compound diagram formed by combining each component D Xa of D la with each 
component Df h of Z) 24 ; where two components are inconsistent, we do not obtain a 
corresponding component in D. In forming D Xa * Df , the number of spiders in a 
zone z is equal to the maximum of the numbers of spiders in the corresponding zones 
of D Xa and D 2h , and z is shaded if and only if at least one of the corresponding zones 
in D- a or D 2h is shaded - see [8] for details. 
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The associativity of * allows us to define the combination of the components of a 
multi-diagram A = {D l , D 2 , . D " } unambiguously as D* = D x * D 2 * ... * D". If A is 
inconsistent, the result will be no diagram; D* is only defined when A is consistent. A 
test for the consistency of A is to try to evaluate D*. Note that there exist inconsistent 
multi-diagrams each of whose proper subsets are consistent. 

Rule 10: The rule of inconsistency. Given an inconsistent multi-diagram A, we may 
replace A with any multi-diagram. 

Rule 11: The rule of combining diagrams. A consistent multi-diagram A = {£>', D 2 , 
. .., D"} may be replaced by the combined diagram D* = D x * D 2 * ... * D". 



2.7 Soundness and Completeness 

D' is a consequence of D, denoted by D t= D', if every compliant model for D is 
also a compliant model for D' . A rule is valid if, whenever a diagram D' is obtained 

from a diagram D by a single application of the rule then D t= D' . We write A 1- D' to 
denote that diagram D' is obtained from multi-diagram A by applying a sequence of 

transformations. We write D I- D' to mean {D} I- D', etc. 

For space reasons, we omit the proofs of the validity of rules 1 to 11. These rules 
are similar to those of the Venn-11 system given in [13] and the proofs are fairly 
straightforward. It can be noted that rules 5, introduction of a contour, 6, splitting 
spiders, 7, excluded middle, and 11, combining diagrams do not lose any semantic 
information; this is useful for the proof completeness. 

Theorem 1 Soundness Theorem Let A be a multi-diagram and D' a diagram. 

Then A 1- D' => A 1= D' . 

The result follows by induction from the validity of the rules. To prove 
completeness we show that if diagram D' is a consequence of multi-diagram A, then A 
can be transformed into D' by a finite sequence of applications of the rules. That is, 

A 1= D' => A I- D' . The proof of the completeness theorem, together with an 
explanation of the strategy of the proof, can be found in [8]. 

Theorem 2 Completeness Theorem Let A be a multi-diagram and let D' be a 
diagram. Then A i= D' => A 1- D'. 



3 Extending the Notation 

In this section we introduce new notation into the SD2 system, extend the 
transformation rules to include the new notation and show that the extended system is 
sound and complete. 
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In fact the new syntactic elements we introduce do not increase the formal 
expressiveness of the system as a whole. However, they do increase the expressive 
power of unitary diagrams so that information can be represented more compactly and 
naturally using the extended notation. 



3.1 Extending the Notation 

The notion of a strand was introduced into spider diagrams (see [3]) to provide a 
means for denoting that spiders may represent the same element should they occur in 
the same zone. A strand is a wavy line connecting two nodes, from different spiders, 
placed in the same zone. The web of spiders s and t, written £(s, t ), is the union of 
zones z having the property that there is a sequence of spiders s = s 0 , Si, s 2 , ... ,s n = t 
such that, for i = 0, ... , n— 1, s t and s i+ \ are connected by a strand in z. Two spiders 
with a non-empty web are referred to as friends. Two spiders s and t may (but not 
necessarily must) denote the same element if that element is in the set denoted by the 
web of s and t. In Figure 4, it is possible that if the elements denoted by s and t 
happen to be in A n B then they may be the same element. 



A 

/ s 


B 

r \ \ 


V 


V 5* ) ‘ / 



D 



Figure 4 

A Schrodinger spider is a spider each of whose feet is represented by a small open 
circle. A Schrodinger spider denotes a set whose size is zero or one: rather like 
Schrodinger’ s cat, one is not sure whether the element represented by a Schrodinger 
spider exists or not. Because of this, a Schrodinger spider in a non-shaded region does 
not assert anything; however, in shaded regions they are useful for specifying bounds 
for the cardinality of the set denoted by the region. They are also useful in 
representing the negation of a diagram (see next section). The set of Schrodinger 
spiders in diagram D is denoted by S* = S*(D). We also let T*(r) denote the set of 
Schrodinger spiders touching region r and S*(r) denote the set of complete 
Schrodinger spiders in r. From Figure 5, we can deduce \A- B\<\, 1 < \A n B\ < 2, 
\<\A\<2 and \B-A\< 1. 




U 



Figure 5 
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SD2 is based on Venn diagrams; that is, all possible intersection of districts must 
occur. In general, spider diagrams are based on Euler diagrams, in which information 
regarding set containment and disjointness is given visually (in terms of enclosure and 
exclusion). A spider diagram based on a Venn diagram is said to be in Venn form', 
otherwise, it is in Euler form. Figure 6 shows a spider diagram in Euler form. 




Extending SD2 by including strands and Schrodinger spiders and basing it on 
Euler diagrams we can express the system’s semantics as the conjunction of the 
following conditions. 

Spiders Condition: A non-Schrodinger spider denotes the existence of an element in 
the set denoted by its habitat and the elements denoted by two distinct non- 
Schrodinger spiders are distinct unless they fall within the set denoted by the zone in 
the spiders’ web: 

3xi,...,3x„ •[ A x i e A (x t -x • => _V Xj ,Xj eT^z))] 

, 4„ J=l.„ 

i*j 

where S= {ji, .... s„}. 

Plane Tiling Condition: All elements fall within sets denoted by zones: 

U^)= u 

zeZ 

Shading Condition: The set denoted by a shaded region contains no elements other 
than those denoted by spiders (including Schrodinger spiders): 

A|W|<yr)| + |rV)| 

reR* 



3.2 Rules of Transformation for Extended Notation 

We can adapt and extend the rules of transformation given in section 2.6 to include 
rules involving the extended notation. 

Adapted Rule 6: Splitting spiders. If a unitary diagram D has a spider s formed by 
two or more feet then we may remove a leg of this spider and replace D with a 
connection of two unitary diagrams D\-D 2 , each containing a different component of 
the split spider. If splitting a spider disconnects any component of the ‘strand graph’ 
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in a zone, then the components so formed should be reconnected using one or more 
strands to restore the original component. The rule is reversible: if a compound 
diagrams contains diagram D\ and D 2 as just described, then D\ and D 2 can be 
replaced by diagram D. 




d A A 



Figure 7 

Rule 12: The rule of strand equivalence. A unitary a diagram D (that is, a unitary 
diagram in which each spider is single-footed) containing a strand in a zone can be 
replaced by a pair of connected unitary diagrams D\ and D 2 which are copies of D. In 
D\ the strand is deleted and in D 2 the two spiders connected by the strand are deleted 
and replaced by a single-footed spider in the zone originally containing the strand. 
Again, the rule is reversible. 




Rule 13: The rule of Schrodinger spider equivalence. A unitary diagram D 
containing a Schrodinger spider strand can be replaced by a pair of connected unitary 
diagrams D\ and D 2 which are copies of D. In the Schrodinger spider is deleted 
and in D 2 the Schrodinger spider is deleted and replaced by non-Schrodinger spider. 
Again, the rule is reversible. 




Figure 9 



Rule 14: The rule of equivalence of Venn and Euler forms. For a given unitary 
diagram in Euler form there is an equivalent unitary diagram in Venn form. 

Figure 10 shows equivalent diagrams in Euler (left) and Venn form. 
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3.3 Soundness and Completeness of Extended System 

The validity of the rules stated above is given in Theorem 3. 

Theorem 3 If D' is derived from D by a sequence of applications of adapted rule 6 
or rule 12 or rule 13 or rule 14, then D 1= D' and D' (= D. 

The proof is omitted for space reasons. The soundness of the extended SD2 system is 
derived by induction from the validity of each of the rules. 

Theorem 4 Completeness Theorem for extended SD2 system. Let A be an 

extended SD2 multi-diagram and let D' be an extended SD2 diagram. Then 

A 1= D' => A I- D'. 

Proof. Assume that A 1 = D'. We apply rule 14 to each component of each diagram in 
A, so that each diagram is in Venn form. Next, we apply adapted rule 6 repeatedly 
to each diagram until all the spiders are single footed. We then apply rule 12 
repeatedly until we have removed all strands and then rule 13 repeatedly to 
remove all Schrodinger spiders. The resulting multi-diagram A SD1 is a set of SD2 
diagrams. We apply the same strategy to D' to produce D' sd2 an SD2 diagram. 
By Theorem 3 A SD1 t= A, and D' 1= D' sd 2 . Hence by transitivity, A SD1 1= 
D' sd1 . So, by Theorem 2 (completeness of SD2), A SD1 I— D' sd1 . For each 
diagram D sd1 in A SD1 there is a diagram D in A such that D \- D sd1 , so we 
have, A I- D ' sd 2 . Each of the rules adapted rule 6, 12, 13, 14 is reversible, so 
D' sd 2 \- D'. Hence, by transitivity, A 1- D'. 



3.4 Derived Reasoning Results 

In this section we enrich the system by providing additional results for reasoning 
with diagrams containing strands or Schrodinger spiders. Several of the results are 
extensions of the given rules to include diagrams with strands. Each lemma is 
illustrated in a figure immediately following the lemma. Their proofs are omitted. 

Lemma 1 (Rule 2): Erasure of a spider. We may erase a complete spider on any 
non-shaded region and any strand connected to it. If removing a spider disconnects 






On the Completeness and Expressiveness of Spider Diagram Systems 37 



any component of the ‘strand graph’ in a zone, then the components so formed should 
be reconnected using one or more strands to restore the original component. 




Lemma 2 (Rule 5): Introduction of a contour. A new contour may be drawn 
interior to the bounding rectangle observing the partial-overlapping rule: each zone 
splits into two zones with the introduction of the new contour. Each foot of a spider is 
replaced with a connected pair of feet, one in each new zone. Likewise, each strand 
bifurcates and becomes a pair of strands, one in each new zone. Shaded zones become 
corresponding shaded regions. 




U 



Figure 12 



Lemma 3 (Rule 7): Rule of excluded middle. If a unitary diagram D has a non- 
shaded zone z, then we may replace D with D\-D 2 . D\ is copy of D where zone z is 
shaded and Di is copy of D where zone z contains an extra single-footed non- 
Schrodinger spider. 




D 



u 




Figure 13 



Lemma 4: Elimination of a strand I. Let D be a unitary diagram containing a 
single-footed non-Schrodinger spider s in an unshaded zone z connected by a strand to 
a non-Schrodinger spider t whose habitat is unshaded. Then D is equivalent to a 
diagram D' in which the spider t has been deleted. 
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Lemma 5: Elimination of a strand II. Let D be a unitary diagram containing a 
single-footed non-Schrodinger spider s in an unshaded zone z connected by a strand to 
a non-Schrodinger spider t. Then D is equivalent to a diagram D' in which the part of t 
lying in an unshaded region is deleted and that part of t lying in a shaded region is 
replaced by a Schrodinger spider. 




Example. In this example we give a diagrammatic proof that the combination of D\ 
and D 2 is equivalent to D. 




First, transform D\ into its equivalent a SD2 diagram by lemma 5, elimination of a 
strand, rule 14, Venn-Euler equivalence, and rule 13, Schrodinger spider equivalence, 
as follows 




Combining D lc and D 2 by rule 1 1 we obtain D'. 
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U 



D' 



Again, by rule 13, Schrodinger spider equivalence, we obtain a unitary diagram D". 




D" 



u 



Finally, since we can transform the Euler form D into the Venn form D" above, we 
can complete proof and obtain D from D" . 

It is worth noting that, given an SD2 diagram in Euler form, the transformation to 
its corresponding Venn form is algorithmic. However, in general transforming from 
Venn to Euler forms is not mechanical. 



4 Negation 

One of the important properties of the SD2 system is that it is syntactically rich 
enough to express the negation of any diagram D in a reasonably natural manner. We 
describe the construction of the negation of D, a diagram which may include 
Schrodinger spiders, in several stages as follows. 

(i) D is an a unitary diagram with n zones which are shaded or contain spiders. The 
negation of D gives a (compound) diagram with m components (»/ > /?). Any 
non-shaded zone z with p spiders gives a unitary component where its 
corresponding zone zj is shaded and contains p — 1 Schrodinger spiders. Any 
shaded zone z with q Schrodinger spiders and r (r > 1) non-Schrodinger spiders 
gives two unitary components where its corresponding zones zi and z 2 contain 
q + r + 1 non-Schrodinger spiders and r - 1 Schrodinger spiders respectively and 
z 2 is shaded. (When r = 0 we obtain a single unitary component whose 
corresponding zone Zj contains q + 1 non-Schrodinger spiders). If D is an a 
unitary diagram where no zone is shaded or contains spiders, its negation is any 
inconsistent multi-diagram. 

(ii) D is a compound diagram with n a unitary components. The negation of D gives 
a multi-diagram formed by n (compound) diagrams being each member of the 
collection the negation of each a unitary component as in case (i) 

(iii) D is any (compound) diagram. We transform it into its a diagram D a and negate 
D a as in (ii). 
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(iv) D is any multi-diagram. The negation of D is equivalent to negate D*, the result 
of combining the components of D, as in (ii). 



Figure 16 illustrates the negation of an a unitary diagram. We use the diagrammatic 
notation D to denote the negation of D. 




D 



u 




A A A 

Figure 16 



5 Conclusion and Related Work 

We have extended the syntax and inference rules of the system of spider diagrams 
we call SD2 and have shown that this extended system is sound and complete. We 
have given a number of derived reasoning rules to aid reasoning in the extended 
notation and shown that we can syntactically give the inverse of any diagram in this 
system. This extended system contains most of the syntactic elements of spider 
diagrams given in [3]. 

Our longer term aim is to prove similar results for constraint diagrams, and to 
provide the necessary mathematical underpinning for the development of software 
tools to aid the diagrammatic reasoning process. 
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Abstract. A key component of computational diagrammatic reasoning 
is the automated interpretation of diagram notations. One common and 
successful approach to this is based on attributed multiset grammars. 
The disadvantages of grammars are, however, that they do not allow 
ready integration of semantic information and that the underlying the- 
ory is not strongly developed. Therefore, embeddings of grammars into 
first-order logic have been investigated. Unfortunately, these are unsat- 
isfactory: Either they are complex and unnatural or else, because of the 
monotonicity of classical first-order logic, cannot handle diagrammatic 
reasoning. We investigate the use of two non-standard logics, namely 
linear logic and situation theory, for the formalization of diagram inter- 
pretation and reasoning. The chief advantage of linear logic is that it 
is a resource-oriented logic, which renders the embedding of grammars 
straightforward. Situation theory, on the other hand, has been designed 
for capturing the semantics of natural language and offers powerful meth- 
ods for modelling more complex aspects of language, such as incomplete 
views of the world. The paper illustrates embeddings of grammar-based 
interpretation into both formalisms and also discusses their integration. 



1 Introduction 

Diagram understanding is a basic prerequisite for diagrammatic reasoning. Thus, 
a key component of computational diagrammatic reasoning is the automated 
interpretation of diagram notations such as state charts, weather maps or struc- 
tural chemical formula. The syntax-oriented approach to diagram understanding 
can be split into two tasks: parsing and interpretation proper [MMAOO]. Pars- 
ing is the process of recognizing the (usually hierarchical) syntactic structure of 
a diagram from a flat representation of its components, while interpretation is 
the process of building a representation of the meaning of the diagram from its 
syntactic structure. 

Computational approaches to syntax-based diagram interpretation are most 
commonly built on multi-dimensional grammars. Grammar-based parsing is rel- 
atively well-understood, having been investigated for over 30 years [MMW98]. 
Grammars can also be used to formalize limited types of diagrammatic reasoning 
by means of diagram transformations. 

Consider reasoning about whether a particular string is in the language of the 
automaton represented by a state transition diagram. As illustrated in Figure 1 



M. Anderson, P. Cheng, and V. Haarslev (Eds.): Diagrams 2000, LNAI 1889, pp. 42—57, 2000. 
(c) Springer- Verlag Berlin Heidelberg 2000 
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we can perform this reasoning by specifying the behaviour of the automaton in 
terms of syntactic transformations of the state transition diagram. Essentially 
such transformations detail an animated execution of the automaton. Note that 
in Figure 1 the remaining input and the current state are denoted by a text label 
that is always moved to below the current state. 

In general, such diagram transformations detail how one diagram can syntac- 
tically be transformed into another diagram that directly and explicitly exhibits 
additional information that is either not present or only implicit and hidden 
in the prior diagram. This can be understood as a form of diagrammatic rea- 
soning by simulation. Unrestricted grammars have been used to express such 
transformation systems [MM97]. 

Multi-dimensional grammars, however, have two significant disadvantages. 
First, grammars do not allow ready integration of other domain information. 
Second, their underlying theory is not well developed, meaning that it is difficult, 
for instance, to prove the equivalence of grammars. 

In part for these reasons, embed- 
dings of grammars into first-order logic 
have been investigated. Unfortunately, 
these embeddings are unsatisfactory. 

The natural embedding in which dia- 
gram elements are modelled by pred- 
icates is unsatisfactory for diagram- 
matic reasoning. This is because the combination of a single, flat universe and 
monotonicity in classical first-order logic 1 means that any piece of information 
is either true throughout the universe or not at all. It is not possible to model 
a transformation in a natural manner if this transformation changes the dia- 
gram in a non- monotonic way, for instance by deletion of an element. Of course, 
this can be achieved in first-order logic by modelling diagrams as complex terms. 
However, this approach leads to a complex and convoluted modelling which does 
not leverage from the underlying logical inference rules. 

In this paper, we investigate the formalization and computational implemen- 
tation of diagram interpretation and reasoning with two non-standard logics that 
are more suitable for these tasks, namely linear logic [Gir87] and situation the- 
ory [SM97] . The chief advantage of linear logic is that it is a resource-oriented 
logic, which renders the embedding of grammars straight-forward. While linear 
logic eliminates a major technical problem (monotonicity) and is able to model 
change over time, it offers no adequate way to model partial and incomplete 
knowledge of the world. In contrast, a sub-divided universe is an inherent notion 
in situation theory. Situation theory was explicitly developed to model natural 
language semantics and discourse. It is therefore a good candidate for modelling 
more complex semantic aspects of diagrammatic reasoning and may also offer a 
better basis to account for cognitive phenomena in diagram perception, such as 
the focus of attention. 




Fig. 1 . State Transition Diagram 



1 Whenever we mention first-order logic in the remainder of the paper we are referring 
to classical first-order logic. 
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The main contributions of the current paper are: We detail the problems 
with embedding multi-dimensional grammatical formalisms into first-order logic 
(Section 3). We exemplify how to naturally encode attributed multiset grammars 
in linear logic so that parsing and transformation can be modelled (Section 4). 
We demonstrate how standard situation-theoretic grammar can be extended to 
capture attributed multiset grammars (Section 5). Finally, we speculate how a 
combination of linear logic with situation theory can provide a powerful inte- 
grated formalism for diagram interpretation and reasoning (Section 6). 

The work described in the paper is very much in progress. As a consequence, 
our argument proceeds mainly by means of examples. We intend to detail the 
formal theory behind these concepts in the future. 

Various researchers have directly used first-order logic for diagram inter- 
pretation, see e.g. [RM89,HM91,Mey97]. In [Haa98] description logic [WS92] is 
used as the formal basis, while [GC96] is based on a specialized spatial logic. 
Explicit embeddings for multi-dimensional grammars into first-order logic are 
given in [Tan91,HMO91,Mar94,MMW98,Mey00]. The current paper is the first 
to critically evaluate the use of first-order logic for diagram interpretation. To 
the best of our knowledge, it is also the first to look at diagram interpretation 
and transformation in terms of linear logic. The use of situation theory for for- 
malizing diagrammatic reasoning is not new. Indeed, some of the most interest- 
ing accounts of diagrammatic reasoning have been given in a situation-theoretic 
setting [Slii96b,Wol98,Shi96a], However, the work in these papers focuses on 
particular notations [Shi96b,Wol98] or particular cognitive aspects [Shi96f ] and 
they do not attempt to provide a general framework for diagram interpretation 
or diagram transformation. 

2 Diagram Interpretation with Attributed Multiset 
Grammars 

As we have indicated, computational approaches to diagram interpretation are 
usually based on multi-dimensional grammars. Here we review a particular 
type of attributed multiset grammars, termed constraint multiset grammars 
(CMGs) [HM091], which have been used by a number of researchers for rea- 
sonably complex tasks such as the interpretation of state diagrams and mathe- 
matical equations. 2 In [Mar94] a precise formal treatment is given, and we review 
only the basic notions here. 

CMGs can be viewed as an extension of classical string grammars. In contrast 
to strings, which are built by linear concatenation of tokens, diagrams must be 
defined in terms of more complex spatial arrangements of graphical tokens. In 
CMGs, this is achieved by equipping the tokens with geometric attributes and 
by making the spatial relations explicit in the productions. Productions have the 
form 

2 We note that these example languages can all be unambiguously interpreted on the 
basis of their syntax only: Thus interpretation is simpler than in the general case 
since contextual information need not be considered. 
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P ::= Pi, . . . , P n where ( C ) {E} 

indicating that the non-terminal symbol P can be recognized from the sym- 
bols Pi, P n whenever the attributes of P\, . . . , P n satisfy the constraints C. 
The attributes of P are computed using the assignment expression E. The con- 
straints enable information about spatial layout and relationships to be naturally 
encoded in the grammar. The terms terminal and non-terminal are used analo- 
gously to the case in string languages. The only difference lies in the fact that 
terminal types in CMGs refer to graphic primitives, such as line and circle , in- 
stead of textual tokens and each of these symbol types has a set of one or more 
attributes, typically used to describe its geometric properties. A symbol is an in- 
stance of a symbol type. In each grammar, there is a distinguished non-terminal 
symbol type called the start type. Interpretation of the diagram is achieved by 
equipping the productions with additional semantic attributes in which the in- 
terpretation is built up during the parsing process. 

As a running example for diagram interpretation, consider the language 
of state transition diagrams. A typical example was given in Figure 1. The 
terminal symbol type declarations for this grammar and their associated at- 
tributes are: arrow (start:point,mid:point,end:point), text(mid:point,label:string), 
circle (mid:point, radius :real) . The non-terminal symbol types for these diagrams 
are std(ss:states,ts:transitions), arc(start:point,mid:point,end:point,label:string), 
state(mid:point,radius:real,label:string, kind: string) and transition (start: string, 
tran: string, end: string) where std is the start type. 

As an example of a production in a CMG consider that for recognizing a 
final state. This is made up of two circles Cl and C2 and text T satisfying three 
geometric relationships: the mid-points of the circle Cl and the circle C2 are the 
same; the mid-points of the circle Cl and the text T are the same; and C2 is 
the outermost circle. Note how the semantic attributes kind and label are used 
to construct an interpretation. The production is: 

S:state ::= Cl:circle,C2:circle,T:text where ( 

Cl. mid = C2.mid and Cl. mid = T.mid and Cl. radius < C2. radius ) { 

S.mid = Cl. mid and S. radius = C2. radius and 
S. label = T. label and S. kind = final } 

Using similar productions we can define normal states and start states as 
well as an arc which is an arrow with text above its midpoint. CMGs also 
include context-sensitive productions. Context symbols, i.e. symbols that are 
not consumed when a production is applied, are existentially quantified in a 
production. The following context-sensitive production recognizes a transition: 

T:transition ::= A:arc 

exist Sl:state,S2:state where ( 

OnCircle( A . start,Sl. mid,Sl. radius ) and OnCircle(A . end, S2. mid, S2. radius ) ) 
{T. start = SI. label and T.tran = A. label and T.end = S2. label } 

Grammatical formalisms have also been used for specifying limited forms 
of diagrammatic reasoning. The key is to regard diagrammatic reasoning as a 
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syntactic transformation between diagram configurations. Unrestricted (type-0) 
variants of graph and multiset grammars can be used to specify these diagram 
transformations. As an example, consider the following production which spec- 
ifies how a single character recognition step in a finite state automaton is exe- 
cuted: 



T:transition, Sl:state, S2:state, Ll:text ::= 

T:transition, Sl:state, S2:state, L0:text where ( 
fi,rst(LO.label)=T.tran and LO.mid=Sl.mid - (0, SI. radius) and 
T.start= SI. label and T.end=S2. label ) { 

Ll.mid=S2.mid - (0,S2. radius) and LI. label=rest(LO. label) } 



3 Embedding Grammars into First-Order Logic 



The great advantage of multi-dimensional grammatical formalisms is that con- 
siderable research has focused on finding efficient parsing algorithms. However, 
they also have two severe limitations. The first is that they are a relatively weak 
computational formalism which is not well-suited to encoding non-diagrammatic 
domain information. The second limitation is that multi-dimensional grammars 
are not well-understood. Parsing with them is understood, but many of their 
formal properties are not. In particular, there is no deductive calculus for gram- 
mars. Therefore we have no methods to prove important properties, such as 
equivalence of two grammars. 

For these reasons, there has been interest in embedding attribute multi- 
set grammars into first-order logic. The motivation is that first-order logic al- 
lows ready encoding of non-diagrammatic domain information and that it will 
allow one to prove properties of the embedded grammars. There have been 
two main approaches: In the first approach the diagram is encoded as a sin- 
gle term which models a hierarchical collection of the graphical objects in the 
diagram. In contrast, the second approach encodes the graphical objects di- 
rectly as predicates. The first approach has been demonstrated in different 
variants in [Tan91,HM091,Mcy97,MMW98], while the second approach is used 
in [HM91,Mar94,Mey00]. 

The first approach models parsing explicitly in the logic, much as parsing of 
string grammars is encoded in the DCG transformation used in Prolog [PW80]. 
The idea is to model a diagram as a single hierarchical term. The key predicate 
is reduce(D, D') which holds if diagram D can be reduced to D' using one of 
the productions in the grammar. To handle spatial conditions, we assume a first- 
order theory C which models the relevant aspects of geometry. C is assumed to 
be an additional set of axioms in our inferences. 

For the state transition diagram grammar we can define reduce f s {D 1 D') 
which holds if D' can be obtained from D by recognizing a final state: 
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reducef s (D, D 1 ) <-> 3{D,D'}. 

circle^C^Cl^) € D A circle(C 2 mzd ’ ^ radius) ^ D A 
text(T mid ,Tiabei) D A 



r 1 

w mid 



^ mid ^ ^ mid ^mid A C ra( n us ^ C radius ^ 



$ mid — C mid ^ $ radius — C radius ^ $ label — ^ label ^ $ kind — final A 

D' = D\ { circle(C l mid , C^ diua ), circle(C 2 mid ’ (-'radius')-’ ^ ex ^{Tmid, Fjabez)} 

l±i { State) S rn j,d , S ra dius . Si abe l , Ski n d ) } 



where we use the Prolog convention of using identifiers starting with an upper- 
case letter for variables and those starting with a lowercase letter for predicates 
and functions, the functions l±l, \ and £ work on multisets, and 3{£>, D 1 } indi- 
cates existential closure over all variables but D and D' . 

It is also straightforward to model simple diagram transformation in this 
way. Our example transformation is mapped to: 

transform(D,D') <-> 3{D,D'}. 

state(S mid , Sr adius , S label , S kind ) £ D A 

state(S 2 mid , S radius ’ ^ label ’ ^ kind ) ^ ^ A 

trans(T st art 7 Ttrani T en d) £ D A text(L° mid , L® abel ) £ D A 
first{L° label ) = Ttmn A L° mid = - ( 0 , S] ud ius ) A 

Tstart = Sjabel A T en d = Sf abel A 

L mid = S mid ~ (0- S radius ) A L L e Z = re S t(T label ) A 
D' = D\ { text(L° mid , L? a6 J} w {tea;t(L^ d , T} a6e/ )} 



Certainly this encoding of parsing and diagram transformation is faithful — it 
exactly captures rewriting with a CMG production. However, the drawback is 
that the encoding does not greatly leverage from first-order logic. Ideally we 
would like reduction in the grammar to be directly modelled by implication in 
the logic. 

This idea is the basis for the second approach in which elements in a diagram 
are identified with predicates rather than terms. For the state transition diagram 
grammar we can define the predicate state which holds if D' can be obtained 
from D by recognizing a final state: 



State(S m id, S ra diusi ^ label , ' 3 {S m id . S ra dius 5 S label: Skind}- 

cirde (CLd’ C radius) A cirC H G mid’ C radius) A text(T mid ,Tl abel ) A 



C 1 

w mid 



O mid A O' mid T m id A @ radius A C ra dius A 



Smid — C md A S radius — O radius A S label — T[ abe i A S^ind — final 



The advantage of this embedding is that implication directly corresponds to 
reduction. The disadvantage is that there is a fundamental mismatch between 
implication in first-order logic and grammatical reduction. The problem is that 
first-order logic is monotonic — something that has been proven true can never 
become untrue. Therefore, there is no way of determining if a particular element 
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in the diagram has already been reduced or if all elements in the original dia- 
gram have been reduced to the start symbol. Thus parsing cannot truthfully be 
captured in this form. 

Even more seriously, monotonicity means that there is no way to model 
diagram transformation, since this requires the diagram to change, with valid 
propositions becoming invalid. Consider the example of the state automaton. An 
execution step requires moving the current input marker from the current state 
to the next state. This clearly requires removing it from its current position, 
but, because of monotonicity of inference, this is impossible to model in classical 
first-order logic. 

4 Embedding Grammars into Linear Logic 

As we have seen, direct modelling of CMGs is not possible in first-order logic. 
The problem is that first-order logic is designed to model eternal laws and stable 
truths [Gir9 1] : If X is true and X — >• Y then Y is true and X continues to hold. 
However, in many physical situations, implication is better understood as an 
action which consumes the resource X to make Y. For instance, we can use $10 
to buy a (mediocre) bottle of wine, but after the purchase the $10 is gone. Linear 
logic was introduced by Girard [Gir87] in 1987 to formalize such implication. 
It is a logic of resources and actions and has been widely used in computer 
science to formalize such diverse things as Petri nets and functional language 
implementation. 

The notation X — ° Y (linear implication) is used to indicate that resource X 
can be consumed to give Y . As an example, if A is “$10” and B is “bottle of 
wine” then A B captures the rule that we can purchase a bottle of wine for 
$10, but once we do so the money is gone. Clearly, — o provides a natural means 
of formalizing reduction in grammars. The linear connective X ® Y is used to 
indicate that we have both the resource X and the resource Y . If C is a “six-pack 
of beer” then A ® A B ® C states that for 2 x $10 you can buy both a bottle 
of wine and a six-pack of beer. Notice that A is not the same as A g) A — money 
does not just materialize out of thin air. 

The first step of our modelling is to encode the diagram itself. The connective 
<S> precisely captures the required combination of diagram elements since it is 
commutative and associative. In essence it combines resources into a multiset of 
resources, whereas classical first-order logic is essentially set-based. 

Consider the encoding of a diagram consisting only of a final state. An exam- 
ple with arbitrarily chosen concrete coordinates is represented by the following 
linear logic formula: 3 3(Ci ® C“ M <g) C 2 <8> C% u <g> T <S> T att ) 

T is te/xt( r l rn j X i , iifibei ) , 

- Ci is circZe(CG d ,C'* odius ), for * = 1,2; 

- cr is \(C ' mid = (1,1 miCLdius = 2 ); 

3 For simplicity we use the first-order logic notation V and 3 for quantifiers, rather 
than Girard’s notation (A and V, respectively). 
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- CF is \(C* mid = (l,l))®!(C r 2 adius = 1); and 

- T att is \(T mld = (1,1 ))®'.(T label = “a”). 

Note the use of the exponential operator (!) in the attribute formulas. This allows 
unlimited copying of a resource, so it will enable us to inspect the attributes of 
an object as often as we like. 

It is also relatively straightforward to encode a CMG rule in linear logic. 
If S represents a final state, C a circle and T some text, then T ® C 0 C — ° S 
captures that we can recognize a final state if there are two circles and some 
text. As in the case of first-order logic, the requisite geometric relationships can 
be captured by appropriately constraining the geometric attributes. 

This is possible because linear logic strictly subsumes first-order logic. Any 
first-order logic formula, modulo appropriate rewriting of the connectives and 
application of the exponential operator, is also a linear logic formula. Again, we 
assume an axiomatic definition of the underlying theory of geometry. We shall 
use C u to refer to the linear logic encoding of C. 

Consider the CMG production P for recognizing a final state. The linear 
logic encoding of this production, P u is 

V(T <g> Ci 0 C 2 0 R S <g> A) 



T is tCXt{L. m .i d , Isabel ) , 

C i /S cirde ( C Ld’ ^radius)’ for * = !> 2 

S is state(S m id, S rad i usi ^ label) Skind)i 
R (^m*d = Cmid) ® (^mid = Tmid) 0 (Cy ( 
A is \{S m i d = C'^j id )®!(*S'radras = Crodius 

■ label, — Tiaifei^^l^Skind ~ flTlCll'). 



radius — ^radius)’ 



Notice that we consume the circles and the text as well as the geometric re- 
lationships between the circles and text, and generate unlimited copies of the 
attribute assignment to S. 

On the basis of this encoding we can now embed complete CMGs into 
linear logic. Assume that the grammar G has productions Pi,...,P n and 
let P\\ , P// be their encoding in linear logic. Then the linear logic encod- 
ing G u of G is 

!if® 

The exponential ! is required because each is a resource. Without the !, 
it could only be used exactly once in a derivation while !P/ ; means that the 
grammar rule can be copied and used as many times as necessary. 

Clearly, linear logic can also be used to encode diagram transformation by 
unrestricted CMG rules: The animated execution of a state transition diagram 
can, for instance, be defined by the linear rule 



V(T 0 Si 0 ^2 0 L 0 <g> R -o T <g> Si 0 S 2 0 L x 0 A) 



T is trQ,nsition(T s t a rtiTt ra niTen d )i 
- Si is state(SG d ,S l radius ,Si abel ,S l kind 



); 



50 



Kim Marriott and Bernd Meyer 



- Li is text(L l ld , L\ abel ), for i = 0,1; 

- R is (first(L la bel ) = Ttran)®{L { l nid = Sj nid — ( 0 ,Sr ad i us ))®(T s tart = ^ label ) ® 

(T en d = s fabel )i anc ^ 

- A is K L mid = S Ld - ( 0 ^ S radiu S )W(Ljabel = reSt(T label ) . 

We are now set to interpret parsing as deduction in linear logic. The state- 
ment Fi b F2 denotes that the linear formula F2 is a consequence of Fi . Assume 
that we have a CMG G with start symbol s and an initial diagram D , encoded 
in linear logic as D u . Then D is in the language of G if 

C U ,G U b D u -o 3 s. 

Intuitively, this states that, in the context of the theory of geometry and the 
grammar, the initial diagram can be rewritten to s by applying the formulas 
that result from the encoding of the grammar G. 

This idea clearly extends to understanding diagram transformation as linear 
deduction. Assume that we have CMG transformation rules T and an initial 
diagram D\ , with linear logic encodings T u and D u respectively. Then if D\ can 
be transformed to D2 using rules in T and Dlj is the linear encoding of D2, then 

C u ,T n b D l l ^ £> 2 . 

Thus we have shown that we can naturally embed diagram parsing and trans- 
formation into linear logic. But what have we gained by this, apart from an 
alternative formalization? 

One clear advantage over grammars is that, because linear logic is an exten- 
sion of first-order logic, we can encode non-diagrammatic domain knowledge in it 
and integrate this with diagram understanding and reasoning. Thus we have met 
the basic prerequisite for being able to resolve ambiguities and for integrating 
parsing with semantic theories. 

The other big advantage is that linear logic has a well-developed proof and 
model theory that we can leverage from. This can be used to prove abstract 
properties of grammars, for example, that two grammars are equivalent or that a 
particular state transition diagram recognizes precisely a certain class of strings. 
In how far such proofs can be constructed automatically is, of course, a matter for 
future research due to the undecideability of linear logic. However, this situation 
is not different from the use of first-order logic for specification purposes. The 
possibility of constructing proofs also raises the interpretation of diagrammatic 
reasoning as diagram transformation to a new status. Whereas a pure grammar 
encoding allowed us only to reason about a diagram by simulation (execution of 
the transformation), the embedding into linear logic allows us to reason about 
the behavior of diagrams in the abstract. For example, we could construct a 
proof that every state transition diagram in which there is a cycle through a 
start state and an end state will accept words of unbounded length. 

5 Embedding Grammars into Situation Theory 

In this section we briefly introduce situation theory and sketch one way of han- 
dling diagram interpretation in situation theory. As we have indicated in the 
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introduction, the main reason for our interest in situation theory is that, unlike 
first-order logic and linear logic which inherently model a flat world without sub- 
divisions, situation theory is designed to model a partitioned universe composed 
of different “situations” each reflecting the viewpoint of an agent at some point 
in time. This is important because (1) when we view diagrammatic reasoning as 
the process of transforming diagram configurations, it is natural to regard each 
diagram configuration as a separate situation, and (2) it may allow us to account 
for cognitive phenomena, such as the focus of attention, diagrammatic discourse 
etc. 



Situation theory was introduced by Barwise and Perry in 1983 [BP83]. Their 
motivation was to develop a richer, more honest semantic theory for natural lan- 
guage understanding. Since then it has been a fertile research field for logicians, 
philosophers, linguists and cognitive scientists. Situation theory is a complex 
field and we can only sketch the basic ideas here. For a complete introduction 
the interested reader is referred to [SM97]. 

A key insight underlying situation theory is that the truth of a proposition is 
relative to an observed scene at some point in time, i.e. a situation. For instance, 
truth of the proposition that Paul drinks wine is relative to some particular event 
in time and space. In situation theory we write s |= -C drinks, p, wine, 1 3>. This 
is read as: “the inf on, i.e. the unit of information, <C drinks, p, wine, 1 holds 
in situation s” , where the object p is a unique identification for the person named 
“Paul” . The first part of the infon details the relation and its arguments, in this 
case dr ink s(p, wine), while the last argument is its polarity, which is 1 if the 
relation is true, 0 if it is false. 

Situations are designed to handle partial, incomplete information about the 
world. Such incompleteness can arise, for example, from an agent’s limited view 
of the world or when a situation is used to model a limited aspect of the world, 
such as a particular event or a particular slice of space-time. An important aspect 
of situation theory is the ability to relate such partial situations and to combine 
them into more complete situations. Inferences in situation theory therefore exist 
on two different levels: So-called local inference works only within a particular 
situation, whereas global inference combines knowledge from different situations, 
i.e. it works between situations. 

A common approach to formalizing local inference is to use classical logic 
within situations. On the basis of such a logic, important concepts like the co- 
herence of situations can be formalized. Coherence means that infons that are 
logically implied by the infons holding in a situation must also hold in the situ- 
ation. 

As indicated above, one of the motivations for situation theory was to model 
natural language. The definition and analysis of language structures by means of 
grammars is fundamental to such a modelling. Situation-theoretic grammar was 
introduced by Cooper in [Coo90] and later put into a computational framework 
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by Black [Bla92]. We will now sketch Cooper’s approach 4 and show how it is 
straightforward to extend it to handle multi-dimensional grammars. 

The purpose of a natural language grammar is to define the structure and 
meaning of spoken or written linguistic expressions, respectively corresponding 
to parsing and interpretation proper. Cooper’s model starts from an utterance, 
i.e. the act of speaking or writing an expression. To define the meaning of an 
utterance he introduces a relation gr{u, i) which relates an utterance u to its 
meaning given by an infon i. For example, 

qrf Paul smiles . -C smiles, p ; 1 ^>) 

where the underlined text represents the utterance of the underlined expression. 

More precisely, an utterance of the expression e, written as e, can be mod- 
elled as a situation u such that u |= <C use-of, it, e; 1 > where <C use-of, u, e; 1 
indicates that the expression e has been written or spoken in situation it. 5 

There is some subtlety in this treatment of meaning: The relation gr relates 
two different kinds of situations: A situation in which the expression is uttered 
and another situation which is described by this expression. Thus the relation 
gr must be defined so that if the expression u describes situation s correctly and 
gr(u, a) then we have that s |= a. 

One of the key ideas in Cooper’s approach is that compound utterances are 
modelled using hierarchical situations. In a sense the situation hierarchy mirrors 
the expression’s syntax or parse tree. The grammatical structure is captured 
in a special bracket notation for the phrase structure: [iii, . . . ,u n ] x denotes a 
structure of category X consisting of constituents of the categories u\...u n in 
the corresponding order. For example, if our simple English grammar dictates 
that a sentence is made of a noun phrase followed by verb and another noun 
phrase, then the utterance of a sentence s is given by the phrase structure 



[NP 1 ,V,NP 2 ] 



sent 



The notation of utterances is extended to compound expression in the follow- 
ing way: [u±, . . . ,u n \ x means that an utterance X consists of the utterances 
Hi , . . • , u n such that for each i: X |= -C precede, up u; +i; i »• 

Thus, u = [NPi,V, NP 2 } sent is a hierarchical situation such that 



u |= -C category, u , sent; 1 
u \= <C constituents, u, NPi, V, NP 2 ; 1 
u |= -C precede, NPi, V; 1 
u |= -C precede, V, NP 2 ; 1 

4 Cooper presented two versions of situation-theoretic grammar. We will focus on the 
first version, since the extensions with parametric objects that are introduced in 
the second version do not appear to have immediate relevance for diagrammatic 
languages. 

5 Note that u is not well-founded. The capability to model such reflective situations 
is an important feature of situation theory. 
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The phrase structure notation can be interpreted in two different ways: (1) 
It can be read as the analogue of a grammar production, defining the general 
structure of the grammatical category sent or, (2) if the constituents NP\ , NP 2 
and V are instantiated with concrete utterances, it can be read as a concrete 
sentence together with its parse tree. 

The relation gr, which relates such compound utterances to their meaning, 
is defined in terms of relations for each possible type of constituent, such as 
sentence, phrase, verb, noun etc.: gr(u, i) iff sent(u, i) V np(u , i) V v{u, i) V . . . 

As an example, let us inspect the relation sent which is part of gr and defines 
the meaning of a simple English sentence: 

sent( [NP 1 , V, NP 2 ) sent , < r, x, y; 1 ») if np(NPi, x) A v(V, r ) A np (NP 2 , y). 



The definition of the different components of gr must, of course, be anchored 
with a lexicon that defines the usable vocabulary. A lexicon contains one entry 
for each word, defining its grammatical category and meaning. For example, 



n(u,p) if u |= 



use-of : “Paul” 

category : n 



Since u obviously is a use of the word “Paul” this is equivalent to nf Paul . v). 
The semantics of this utterance is therefore defined to be the object p. Note that 
the bracket notation used above indicates that the infons in the brackets are 
functional, i.e. they define a unique value for a feature of the situation. 

To complete the definition of gr we also need rules that create the basic 
phrase structure for the words in the lexicon: If c is a grammatical category 
defined in the lexicon this is given by c([u] ,i) if c(u, i). 

We will now modify the situation-theoretic grammar approach to han- 
dle multi-dimensional languages. The basic requirement is to allow multi- 
dimensional (geometric) connectives instead of only a linear connective (con- 
catenation). As before, we base the modelling on a set of primitive geometric 
shapes. We collect all geometric properties of an object o into a single record 
loc 0 , so for a circle c, for example, we have loc c =< mid c , radius c >. A “graphi- 
cal utterance” can now be defined as above, provided we extend the infons with 
the spatial information. For example, the use of a circle c is thus defined as a 
situation u such that 



u |= -C use-of ’, u , circle, < mid c , radius c >;1 > 

This is abbreviated as circle ^ oc . The definition is directly inline with the stan- 
dard situation theory framework, as spatio-temporal locations for infons are 
already an integral part of basic situation theory. For convenience, we allow 
reference to the components of loc by name, for example loc c .mid is mid c . 

We can build up hierarchical situations corresponding to complex expressions 
in almost the same way as before. The main difference is that we need to model 
the spatial relations explicitly instead of using the precede relation only. This 
can be done by including the spatial relations in the bracket notation for the 
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phrase structure. We also use identifiers for the individual components instead of 
indexing them like Cooper does. Analogously to Cooper’s model, the utterance 
of a final state in our running example can be defined as a situation Uf where 

{ [TV text, Cl: circle, C2: circle] g. state such that 
S.mid = Cl. mid and S. radius = C2. radius and 
Cl. mid = C2.mid and Cl. mid = T.mid and Cl. radius < C2. radius. 

V : C is an abbreviation for Cf oc ^ r and V.a = locy.a. This defines Uf as a 
situation such that 

Uf \= category,Uf, state ; 1 

Uf \= constituents . ut , texfy 0Ci _, circle^ , circle; cc ; 1 > 

Uf [= <€.=,locci.mid,locc 2 .mid\l 
Uf [= <C=, locT-mid, locci-mid; 1 V?> 

Uf \= <^<,locci.radius,locc 2 -radius\l 

Uf \= [S.mid = Cl.mid, S.radius = C2. radius] 

As in the case of linear logic, we require an underlying geometric constraint 
theory C which defines the arithmetic relations, such as equality of points. In 
situation theory parlance, C has the status of “necessary” constraints, i.e. it 
forms a background knowledge about the world as such and C is valid in all 
situations. Note that C does not model inference between situations, but only 
within each individual situation. Therefore it is an extension to the local logic 
that is used for inference within a situation. Assuming that we use some local 
theory which subsumes first-order logic (in particular first-order logic itself or 
linear logic), C can be captured truthfully. 

What remains to be done is to specify the meaning of an utterance by using 
an analogue of the gr relation. Again we introduce a relation for each type of 
constituent. In the case of final states and assuming uf is defined as above we 
have: 

state{uf , -C kind , final ; 1 A -C label, TL] 1 V§>) 

if text ( T : text , <C label, TL\ 1 3>) A circlei Cl : circle ) A circlei C2 : circle ) 

Let us now look at diagram parsing in the context of diagrammatic reason- 
ing. Previously we have outlined how a diagrammatic reasoning process can be 
understood as a chain of diagram transformations t; that successively transform 
a diagram T>q into a new diagram T> n , which directly and explicitly exhibits 
the conclusions of the reasoning process. Each of the successive diagram config- 
urations can be captured in a separate situation S t so that our diagrammatic 
inference chain amounts to a global inference on the level of the situation struc- 
ture. Using the technique described in this section, diagrammatic parsing can 
likewise be regarded as a process of transformation in which the phrase structure 
is gradually built up. This corresponds to modelling the stages of a bottom-up 
parse as individual situations. 
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6 Towards a Hybrid Approach 

We have shown how Cooper’s situation theoretic grammar can be extended to 
capture diagrammatic notations. Embedding the specification of diagrams into 
situation theory gives us a powerful logical formalism for discussing semantic 
and some cognitive phenomenon in diagram interpretation and reasoning. 

In the way we have defined them above, grammars specify a global theory, 
since each reduction step in parsing gives rise to a new situation. However, this 
is not the only way, nor necessarily the best way, to model parsing in situation 
theory. The main problem with this approach is that a formal framework for 
logical inference in situation theory is not yet as well developed as in linear logic. 
In particular, a general syntactic calculus for inference on the global level (i.e. , 
for inferences involving more than one situation) has yet to be developed. This 
is not surprising given the expressiveness of situation theory and its consequent 
complexity, but it means that one cannot readily prove properties of grammars 
if they are encoded in terms of a global theory. 

On the other hand, local inference within a single situation is far better 
understood. We can exploit this in a different approach to performing diagram 
interpretation in situation theory. The idea is to employ global inference only at 
the topmost level to handle the transformations involved in true diagrammatic 
reasoning. Simpler tasks, such as parsing and basic syntactic interpretation, can 
be contracted into a single situation and handled by local inference. We have 
illustrated how these tasks can be formalized in linear logic. It appears possible 
to achieve the integration of both approaches by using linear logic as the calculus 
of local inference within situation theory. 

Such a hybrid approach potentially offers the advantages of both embeddings. 
The use of linear logic for local inference in situation theory could bridge the 
gap between an executable, computationally-oriented approach to diagram inter- 
pretation and a general semantic account of diagrammatic reasoning. However, 
integrating linear logic into situation theory as a local calculus is by no means a 
small endeavour. Channel Theory [BS97], an offspring of situation theory which 
attempts to formalize the collaboration of different types of information manip- 
ulating systems in a single composite system, may turn out to provide a general 
key for this integration as well as for the integration of diagrammatic reasoning 
with other kinds of reasoning. 

We feel that it is well worth exploring the hybrid approach since it has the 
potential to combine an executable logic, proof theory and semantics into a single 
unifying framework for diagram interpretation and diagrammatic reasoning. 
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Abstract. By devising a new reading method for Peirce’s Existential 
Graphs (EG), this paper moves away from the traditional method of eval- 
uating diagrammatic systems against the criteria appropriate to symbolic 
systems. As is well-known, symbolic systems have long been preferred to 
diagrammatic systems and the distinction between the two types of sys- 
tems has not been well defined. This state of affairs has resulted in a 
vicious circle: because the unique strengths of visual systems have not 
been discovered, diagrammatic systems have been criticized for lacking 
the properties of a symbolic system, which, in turn, reinforces the ex- 
isting prejudice against non-symbolic systems. Peirce’s EG is a classic 
example of this vicious circle. 

Logicians commonly complain that EG is too complicated to put to 
actual use. This paper locates a main source of this criticism in the 
traditional reading methods of EG, none of which fully exploits the visual 
features of the system. By taking full advantage of the iconicity of EG, I 
present a much more transparent and useful reading of the Beta graphs. 
I pursue this project by (i) uncovering important visual features of EG, 
and (ii) implementing Peirce’s original intuitions for the system. 



1 Introduction 

One shared view among diagrammatic reasoning researchers is that symbolic 
systems have been strongly preferred to diagrammatic systems throughout the 
history of modern logic and mathematics. This belief clearly presupposes that 
symbols and diagrams are different from each other. However, no clear distinction 
has been made among different kinds of representation systems. I would like to 
focus on a circular relation formed between the prejudice against diagrammatic 
systems and the non-clarified distinction of diagrams from symbols. 

Without a solid theoretical background for the distinction between symbolic 
and diagrammatic systems, we easily overlook different kinds of strengths and 
weaknesses each type of system exhibits. Then, with the general predominance 
of symbolic systems, we tend to try to understand and evaluate diagrammatic 
systems against the criteria of a symbolic system. As a result, without its own 
strengths being discovered, a diagrammatic system has been criticized mainly 
because it lacks the properties of a symbolic system. In turn, this unfortunate 
result, stemming partly from a prejudice against diagrams and partly from an 
unclear distinction between symbols and diagrams, only reinforces the existing 
prejudice against non-symbolic systems. I claim that the reception of Peirce’s 
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Existential Graphic System (EG) is a classic example of this vicious circle, and 
I propose EG as a case study to show how to break this undesirable circular 
relation. 

Extensive research has been performed on this impressive graphic system. 1 
However, many logicians strongly prefer symbolic systems over this graphic sys- 
tem. One of the main reasons for this preference is that Peirce’s Beta graphs have 
been considered to be difficult to read off. In this paper, by uncovering visual 
features of EG (in the second section) and by reviving Peirce’s original intuitions 
for the system (in the third section), I present a new reading algorithm in the 
fourth section. In the fifth section, I illustrate through examples the main dif- 
ferences between this new reading and existing ones. In the conclusion, I discuss 
the scope of this project and further work in this direction. 



2 Conjunctive and Disjunctive Juxtaposition 

The Beta system has four kinds of basic vocabulary: cut, juxtaposition, lines 
and predicate symbols. After a brief introduction of the meaning of cut and jux- 
taposition, I identify overlooked visual properties that represent more syntactic 
distinctions in EG than are made on the traditional approach. I do not introduce 
new syntactic devices into EG, but only observe significant visual differences al- 
ready present in EG. 

A cut represents the negation of whatever is written inside the cut, and 
juxtaposition the conjunction among whatever is juxtaposed. In the following 
example, 




(1) (2) (3) (4) (5) 



Graph (1) is read off as “5 is true,” (2) “S is true and W is true,” (3) “It is not 
the case that both S and W are true,” (4) “Neither S nor W is true,” and (5) 
“It is not the case that neither S nor W is true,” which is the same as “Either S 
or W is true.” 

It has long been believed that Peirce’s EG has only two kinds of syntactic 
operation corresponding to logical connectives in a symbolic system, cut and 
juxtaposition. Therefore, Don Roberts emphasizes the importance of the order 
of negation and conjunction for the translation of the following Peircean graph: 



Zeman, Roberts, Sowa, Burch and Hammer. 
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Notice that we do not read it: ‘Q is true and P is false’, even though Q 
is evenly enclosed and P is oddly enclosed . 2 ... we read the graph from 
the outside (or at least enclosed part) and we proceed inwardly . . .. 3 

Roberts suggests the following passage from Peirce as evidence that this method, 
“a method to which Peirce gave the name ‘endoporeutic ’,” 4 was what Peirce 
originally had in mind: 

The interpretation of existential graphs is endoporeutic, that is proceeds 
inwardly; so that a nest sucks the meaning from without inwards unto 
its centre, as a sponge absorbs water . 5 

I claim that this reading method has prevented us from benefiting from the 
visual power of the system. My main criticism is that this method does not reflect 
visually clear facts in the system. As Roberts points out in the above quotation, 
in the graph it is true that Q is evenly enclosed and P is oddly enclosed. However, 
the endoporeutic reading does not reflect directly this visually clear fact at all, 
and what is worse, it leaves the impression that this visual fact is misleading. To 
put this criticism in a more general way: this method forces us to read a graph 
only in one way. For example, we are supposed to read the above graph in the 
following way: This graph is a cut of the juxtaposition of P and a cut of Q. 
However, there are other possible readings. For example, we might say that this 
graph has two cuts with Q enclosed by both cuts and P enclosed by the outer 
cut only. Or, we might say that this graph has two letters, P and Q, and the 
juxtaposition of P and a cut of Q is enclosed by a cut (hence, P is enclosed by 
one cut and Q enclosed by two cuts). These readings are not reflected directly 
in the endoporeutic reading. 

I suggest that the following two features be interpreted directly in a new 
reading: (i) the visual distinction between what is asserted in an evenly enclosed 
area and what is asserted in an oddly enclosed area, and (ii) the visual fact 
that some juxtapositions occur in an evenly enclosed area and some in an oddly 
enclosed area. These suggestions will be reflected in the new reading method 
presented in the next section in the following way: The juxtaposition occurs in 
an evenly enclosed area is read off as conjunction, and the juxtaposition in an 
oddly enclosed area as disjunction. Also, a negation is added by reading off a 
visual fact that a symbol is written in an oddly enclosed area. 

2 X is evenly enclosed iff X is enclosed by an even number of cuts, and X is oddly 
enclosed iff X is enclosed by an odd number of cuts. (My footnote) 

3 Roberts, 1973, p. 39. 

4 Roberts, 1973, p. 39. 

5 Peirce, Ms 650, quoted by Roberts, 1973, p. 39, n.13. 



Reviving the Iconicity of Beta Graphs 



61 



Let me illustrate how to read off these features through the example Roberts 
cited above . 6 P is written in an oddly enclosed area, and Q in an evenly enclosed 
area. Hence, we obtain —<P and Q. The juxtaposition between P and a cut of Q 
occurs in an oddly enclosed area. Therefore, we easily get a sentence 1 ~>P V Q' 
which does not require any further manipulation . 7 

3 Peirce’s Intuition for Iconicity 

Another syntactic device in the Beta system, a line, called ‘a line of identity,’ 
(henceforth, ‘LI’) denotes the existence of an object, and it can have many 
branches, called ‘ligature.’ For example, the first graph says (i) “Something good 
is ugly,” the second (ii) “Something good is useful and ugly,” and the third (iii) 
“It is not the case that some good thing is not ugly,” which is the same as 
“Everything good is ugly .” 8 



is good 
v “ — is ugly 




is good 
is ugly 
is useful 




In this section, we will discuss Peirce’s three important intuitions related to 
Li’s which make this Beta system graphic, and accordingly, distinct from other 
predicate symbolic languages. I claim that all these three valuable intuitions 
should be reflected in our new reading algorithm. 

3.1 Lines of Identity 

It is well known that relations are harder to represent in a graphic system than 
properties . 9 Hence, Peirce’s first motivation for the Beta system is to represent 
relations graphically . 10 His example is the proposition that A is greater than 
something that is greater than B. After suggesting unsuccessful graphs, finally 
he says 

6 Clause 4 in the new reading in section 4 formalizes this process. 

7 Another problem with the traditional reading is that in many cases we obtain a 
complicated-looking sentence, i. e. one with nested negations and conjunctions, 
which requires further syntactic manipulations. 

8 Two of these examples are borrowed from Roberts 1973, p. 51. 

9 For example, well-known Euler or Venn diagrams are limited to the representation 
of properties. (Refer to Shin 1994.) In the discussion of Venn diagrams, which 
precedes the writing on his EG, Peirce points out this aspect as one of the problems 
of the Venn system: “It [The Venn system] does not extend to the logic of relatives.” 
(Peirce, 4.356.) 

10 “In many reasonings it becomes necessary to write a copulative proposition in which 
two members relate to the same individual so as to distinguish these members.” 
(Peirce, 4.442.) 
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[I]t is necessary that the signs of them should be connected in fact. No 
way of doing this can be more perfectly iconic than that exemplified in 
Fig. 78 [the following graph]. 11 



A is greater than 



“L 



is greater than B 



This syntactic device, i.e. the line connecting two predicates, represents one and 
the same object than which A is greater and at the same time which is greater 
than B. This is why Peirce calls this line a line of identity (henceforth, ‘LI’). 12 
This intuition is captured in the following convention: 

A heavy line, called a line of identity, shall be a graph asserting the 

numerical identity of the individuals denoted by its two extremities. 13 

The introduction of lines adds expressive power to this system: it allows 
us to represent individuals and relations, while his Alpha system cannot. At 
the same time, this device makes a clear distinction between Beta graphs as 
a graphic system and other predicate languages as symbolic systems. While 
tokens of the same type of letter represent the numerical identity of individu- 
als each token denotes, one network of lines represents the numerical identity 
among individuals denoted by each branch. That is, Peirce’s system graphically 
represents the numerical identity by one connected network. Visual clearness is 
obtained by this aspect of the system. For example, in the first-order sentence, 
3x3y[Tall(y) A Love{x, y) AWalk(y) ATeacher(x ) A Smile(y) A Woman(x)\, we 
need to keep track of tokens of x and y more carefully than when the identity of 
each individual is represented by graphical lines in the following Beta graph: 



Woman 



Love 



Walk 

Smile 



Teacher 



Tail! 



3.2 Existential versus Universal Quantifiers 



The most striking aspect of the Beta system compared with other predicate 
languages is that it does not introduce a syntactic device which corresponds to 



11 Peirce, 4.442. 

12 Peirce, 4.442. 

13 Peirce, 4.442. 
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quantifiers of symbolic languages. Then, how can the Beta system be about the 
logic of quantifiers ? 14 Rather than adopting one more syntactic device, Peirce 
relies on visual features which already exist in a graph: 

... any line of identity whose outermost part is evenly enclosed refers to 
something, and any one whose outermost part is oddly enclosed refers 
to anything there may be. 15 

Examples are in order : 16 



Since the outermost part of the line is in an evenly enclosed in the graph on the 
left side, it says that something good is ugly. On the other hand, the graph on 
the right side says that everything good is ugly . 17 

That is, existential and universal quantifiers are represented visually by 
whether the outermost part of an LI is evenly or oddly enclosed, without any 
additional diagrammatic or symbolic object introduced. I take this aspect of the 
system to be one of the most important ones, which makes the system graphic. 

3.3 Scope of Quantifiers 

Graphic systems are in general not linear expressions. In symbolic systems which 
are linear, the scope of syntactic operations is clarified by the linear order of the 
symbols, or sometimes along with the use of parentheses. Now, the question 
in the Beta system is how to represent the order among different quantifiers. 
Fore example, how does this system make a distinction between the proposition 
l \/x3yL(x,yY and the proposition { 3y\/xL{x, y)'l 

Again, Peirce solves this problem in the Beta system by appealing to the 
visual feature about how many times lines’ outermost parts are enclosed: The 
less enclosed the outermost part of a line is, the bigger scope the line gets. As 
an example, let us compare the following graphs : 18 

14 As Roberts starts with his chapter on the Beta system, this system is “a treatment 
of the functional or predicate calculus, the logic of quantification.” (Roberts 1973, 
p. 47.) 

15 Peirce, 4.458. The outermost part of an LI is the part which is the least enclosed. 

16 Borrowed from Roberts 1973, p. 51. 

17 In the graph on the right side, some part of the LI is enclosed by one cut and some 
part by two cuts. Therefore, the part enclosed by one cut is the outermost part of 





the LI. 

18 Roberts 1973, p. 52. 
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'adores — — is a wc 



is Cathoic 





is a woman 



In the first graph, the line whose outermost part is written in an oddly enclosed 
area is less enclosed than the line whose outermost part is in an evenly enclosed 
area. Therefore, a universal quantifier has a bigger scope than an existential 
quantifier. In the second graph, the opposite is true. While ‘Vx3 y(Catholic(x) — > 
( Adores(x,y ) A Woman(y))Y is what the first graph expresses, the second says 
‘3 yVx(Woman(y) A ( Catholic(x ) — > Adores{x,y))).’’ 

It should be noticed that Peirce takes an advantage of a homomorphic re- 
lation between the position of lines and their scopes: If the outermost part of 
line l\ contains every area the outermost part of line I 2 contains, then the reading 
of l\ has a bigger scope than the reading of 1 2 . Without linear order or with- 
out additional graphic notation, Peirce disambiguates the scope relation among 
quantifiers. The way he accomplishes this goal, that is, reading off a simple 
feature visualized in a graph, clearly makes this system more graphic. 

4 New Reading 

In this section, I present a new algorithm for reading Beta graphs, which reflects 
two kinds of juxtapositions in the second section and Peirce’s intuitions discussed 
in the third section. 

For further discussion, let us examine more carefully how Li’s are used in this 
system. In the following two graphs, a simple identity line says that something 
is P and Q\ 



Suppose we need to express that something is P, Q and R. This is where an LI 
needs to branch, as follows: 



p 



Q 




p 



O 






In this case, every branch of one and the same LI functions as one identify 
line, that is, every branch denotes the same object. We call this connected web of 
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branches an LI network. In the above graphs, each graph has one LI network, just 
as a first-order language requires one type of variable under the same scope of one 
existential quantifier. Hence, there seems to be a strong connection between an 
LI in Beta graphs and a variable in first-order languages, and we will implement 
this connection in our reading method, with the following exception. 19 

Suppose we translate the proposition that there is something which is P 
and something which is Q and these two are not identical with each other. 
ll 3x3y(Px AQy Ax ^ y) v is a translation in a first-order language. Two variables 
are used under the scope of two existential quantifiers. However, if we use two 
Li’s attached to predicates P and Q respectively as follows, we do not obtain 
what we want: 



P 



Q 



What this graph says is that something is P and something is Q , i.e. (3 xPx A 
3yQy). That is, it fails to express a non-identical relation between two objects. 
We need to negate an identical relation itself, by using a cut. Any of the following 
graphs does the job: 




P 

Q 



] 5 > 



<€Z 



P 

Q 



p ~ O Q 



Therefore, when an LI crosses an odd number of cuts entirely, the generaliza- 
tion that each LI network refers to the same object is not true. On the contrary, 
we need to bring in (at least) two different objects in this case. If a portion of 
an LI crosses an odd number of cuts, we say that that line is clipped. Notice 
that an LI network may be clipped into more than two parts. Both graphs in 
the following have only one LI network, but the LI network in the graph on the 
left is clipped into two parts, while the LI network in the graph on the right is 
clipped into three parts by one cut: 




R 



We will implement these discussions of LI networks in the second step of 
the reading method presented below. Next, we will write atomic wffs at the 



19 



Another main difference is that an LI denotes an object while a variable does not. 
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end of an LI. In the fourth step, we form a complex wff by reading off two 
kinds of juxtaposition. Finally, when we make a sentence out of this complex 
wff, whether the outermost part of an LI network is evenly or oddly enclosed 
determines whether an existential or a universal quantifier will be adopted. 20 



Reading off Beta Graphs 

1. Erase a double cut. 21 

2. Assign variables to an LI network: 

(a) If no portion of an LI network crosses an odd number of cuts 
entirely, then assign a new variable to the outermost part (i.e. its least 
enclosed part) of the LI networks. 

(b) If a portion of an LI network crosses an odd number of cuts entirely (i.e. 
an odd number of cuts clips an LI into more than one part), then 

i. assign a different type of a variable to the outermost part of each 
clipped part of the network, and 

ii. at each joint of branches inside the innermost cut of these odd num- 
ber of cuts, write Vi = Vj , where Vi and Vj are assigned to each 
branch into which the line is clipped by the cuts. 

3. Write atomic wffs 22 for Li’s: (Let us call it a quasi-Beta graph.) 

(a) For an end of an LI with a predicate, say P, replace P with Pv\ . . . v n , 
respecting the order of hooks, 23 where Vi (or [iq]),. . . , and v n (or [tVi]) 24 
are assigned to the lines hooked to P. 

(b) For each loose end of an LI, i.e. an end without a predicate, write T. 

(c) For an LI which does not get any atomic wff or T, i. e. a cycle whose 
part is not clipped, write T at the outermost part and at the innermost 
part of the LI. 

4. Obtain a complex wff: 

Let G be a simple quasi-Beta graph. 25 The following / is a basic function: 

/(G) = a if G is atomic wff a. 

f(G ) = if G is a single cut of atomic wff a. 

/(G) = T if G is T. 

20 Note that in order to read off available visualinformation of a graph as much as 
possible, we will not erase any part of a graph in the reading process, which makes 
a contrast between Zeman’s and this new reading methods. (Zeman, pp. 79-82.) 

21 A scroll [two cuts, one inside the other] is a double cut if and only if nothing is 
written in the outer area of the scroll [i. e. the area between the inner and the outer 
cuts] except the portions of Li’s which cross the outer area entirely. 

22 The term ’wff’ is an abbreviation for a well-formed formula. 

23 In a clockwise direction. 

24 In order to keep track of one unclipped LI network, sometimes we might want to 
write down variables in brackets. Hence, in the step (5), the location of the variables 
in brackets should be ignored. 

25 A simple quasi-Beta graph is an atomic wff, a single cut of atomic wff, T, an empty 
space or an empty cut. 
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/(G) = -tT if G is a single cut of T. 

/(G) = T if G is an empty space. 

/(G) = -iT if G is an empty cut. 

Now, we extend this function / to / to translate a quasi-Beta graph into a 
complex wff: 

(a) /(G) = /(G) if G is a simple quasi-Beta graph. 

(b) 7([[G]]) = /(G)/ 6 

(c) /(Gi . . . G n ) = f(Gi) A ... A /(G„). 

(d) /([G 1 ...G n ]) = /([G 1 ])V...V/([G„]). 

5. Obtain sentences. For each variable Vi in the wff obtained by the previous 
step, 

(a) if Vi is written in an evenly enclosed area in step 2, then add 3t>, im- 
mediately in front of the smallest subformula containing all occurrences 
of Vi, 27 and 

(b) if Vi is written in an oddly enclosed area in step 2, then add \/vi im- 
mediately in front of the smallest subformula containing all occurrences 
of Vi. 28 and 

We will illustrate how each clause works through examples. 

Clause 1 tells us to make the following changes: 



p 




==> p a 







Clause 2(a) respects Peirce’s intuition on LI. That is, a continuity of an 
identity line, however it may branch, visualizes a single object. 2(a) is easily 
applied to get the following stages: 



p 



X 




Q 

R 





But, there is an exception where this intuition breaks down. This is what 
clause 2(b) is about. This clause suggests that a different variable be assigned 
to each part: 



26 In this linear notation, cuts are represented by brackets. 

27 If there is no subformula containing Vi, this step is skipped. 

28 If there is no subformula containing Vi, this step is skipped. 
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P 







Q 



Both in 2(a) and 2(b), it is important to assign a variable to the least enclosed 
part. The location of each variable will play a crucial role when we assign a 
quantifier. Suppose the following graph is given: The first cut clips this network 



P 





into two parts. 29 Two variables, x and y, should be assigned to the least enclosed 
part of each clipped part. Hence, in the following, the first one is correct, while 
the second is not: 



P 



X 




y 





Notice that clause 2(b) has one more requirement, that is, to write down 
Vi = Vj inside the cut. This identity statement represents the visual fact that 
these two clipped portions are connected with each other. The following are the 
result of this clause being applied: 




R 



Let’s move to clause 3. For each end of an LI network, if there is a predicate P, 
then replace this predicate with atomic wff Pv\ . . . v n , and if there is no predicate, 
then write T. The following shows how this rule is applied: 




The next step, clause 4, shows a departure from Peirce’s endoporeutic read- 
ing which all the existing reading methods follow. This endoporeutic reading 
prevents us from implementing Peirce’s visual distinction between universal and 
existential quantifiers. Since all the existing methods for reading off EG adopt 
the endoporeutic method without any exception, we should not be surprised to 



29 



Notice that the second cut does not clip this line. 
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find out that no current method could reflect Peirce’s original intention for this 
insightful visual distinction. 

Step 5 tells us how to obtain a final sentence out of the complex wff formed 
in the previous step, by reading off a visual distinction between some and all. 
Let’s recall that in the step 2, variables are written at the outermost parts of 
Li’s. Depending on whether a variable is written in an evenly enclosed or oddly 
enclosed area, quantifier V or 3 is adopted respectively. 

Suppose we have the following quasi-Beta graph by steps 2 and 3: 



Variable x is written in an evenly enclosed area and y in an oddly enclosed 
area. Therefore, step 5 tells us to introduce quantifiers to obtain 3x[Px A\/y(x = 
V V -> Qy ) A Rx). 

5 Comparison among Different Readings 

Both Zeman’s and Roberts’ contribution to this subject cannot be overempha- 
sized. 30 While both of them provided brilliant methods for reading off EG, they 
accomplished their goal by taking very different routes, which result from their 
different emphases. Zeman aimed to come up with a formal and comprehensive 
translation algorithm for the system. By contrast, Roberts was interested in pre- 
senting Beta graphs at a more informal and intuitive level, which he achieved 
by a close reading of Peirce’s manuscripts. Since space is limited, I will illustrate 
differences among these different methods through several examples. 

For the following Beta graph, 31 




First, function / in clause 4 is applied: 



f(Px[[x = y}Qy}Rx) = f(Px) A /([[ x = y]Qy ]) A f{Rx) 



by (c) 
by (d) 
by (b) 
by (a) 



= /(Px) A (/([[x = y]])V f([Qy})) A /(Px) 
= f(Px) A (/(x = y) V f([Qy})) A /(Px) 

= Px A (x = y V -i Qy ) A Rx 




Zeman’s reading algorithm suggests the following process: 




Break the joint to write L3 Assign variablesto Li’s Wrtie up atomic wffs 



Erase Li’s 



30 Zeman, Chapter 2 and Roberts 1973, Chapter 4. 

31 This graph says “Anything that is P is both Q and R. 
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Now, we obtain the following formula from the rightmost graph: 

->(Px A L 3 xyz A ~>{Qy A Rz)) 

By adding existential quantifiers in front of the smallest subformula, we ob- 
tain the following sentence: 

-3x(Px A 3y3z(L 3 xyz A ~>{Qy A Rz))) 

By replacing a temporary predicate, L 3 , the following sentence is the final trans- 
lation of the above Beta graph by Zeman’s reading: 

-i3x{Px A 3y3z(x = y A y = z A ~^{Qy A Rz))) 

This quite complicated looking formula is logically equivalent to “Wx(P x — > 
(Qx A Rx)).” 

Now, our new algorithm comes up with the following process: 




By 2(a) By 3(a) 



->Px V ( Qx A Rx) by 4 

\/x[~<Px V ( Qx A Rx)] by 5 

This reading process is simpler in the following sense. First, Zeman’s reading 
adopts three types of variable while the new algorithm only one. Hence, Zeman’s 
algorithm, unlike our new one, does not read off the visual fact that this Beta 
graph has one ( not three) unclipped network of LI. Second, Zeman’s reading 
method never has a chance to read off an important iconic feature about a dis- 
tinction between universal and existential statements, but only gets existential 
quantifiers. Hence, universal statements are always expressed as a nested rela- 
tion between negation and existential quantifiers. For these reasons, not only the 
final translation sentence is rather complicated looking, but also Li’s become dis- 
pensable, which is why Zeman’s method erases them in the process. This is quite 
ironic when we recall that the visualization of identity is one of the strengths of 
EG. While Zeman provides us with a very impressive and comprehensive read- 
ing method of this system, he fails to read off important iconic features of this 
graphic system. 

Roberts’ approach is quite different from Zeman’s. Roberts respects Peirce’s 
intuition about what Li’s are supposed to mean in the system: Neither a break- 
ing point nor an additional predicate is introduced. 32 The numerical identity is 
emphasized as the interpretation of an LI, which works very well in some cases. 
However, how does Roberts’ reading handle the cases in which an LI passes 
through an odd number of cuts? The price Roberts’ project had to pay is to 
categorize certain kinds of graphs under special cases and stipulate them rather 
than covering them under a general reading method. 33 These are some of his 
special cases: 

32 Recall that Zeman’s reading requires certain Li’s to be broken into parts and tem- 
porary predicates, L 1 , L 2 and L 3 , to be introduced. 

Roberts 1992, pp. 645-646. 
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Under the heading Special Cases, Roberts gives us the following interpretations 
of the above graphs: 



Literally, this [graph (1)] means “There is an object and there is an 
object and it is false that these objects” - denoted by the two unenclosed 
parts of the line - “are identical.” By means of this graph, we can say 
such things as “Some woman has two husbands” ([graph (3)]). Figure 
28 [graph (2)] means “There is something, and it is false that there is 
something else non- identical to it” - that is, “Something is identical with 
everything.” . . . Figure 31 [graph (4)] expresses the Unitarian theology 
as summarized by Whitehead: “There is one God at most.” 34 



Zeman’s comprehensive reading method does not treat these as special cases. 
After a broken points and temporary predicates being introduced, we obtain 
rather complicated-looking sentences as the translations of these graphs: 

(1) : 3x[x = x A 3y(x ± y y = y)\ 

(2) : 3x[x = x A y(x ± y A y = y)\ 

(3) : 3x3y[x ^ y A 3x3u3v(x = u A u = v A HusbandOf(x,z) A 
HusbandOf(y,v) A W oman{u))\ 

(4) : -i3x3y(God(x) A God(y) A x j(j) 

Unlike Roberts’ method, our new reading algorithm covers these cases without 
treating them as special. At the same time, this new method yields the result 
more easily with simpler sentences than Zeman’s. According to our algorithm, 
after applying clauses 1 and 2, each graph is transformed into the following: 




According to clause 3, the following complex wffs are obtained: 

From (1)': T Ax/j/AT. 

From (2)': T A (x = y V -iT). 

From (3)': x/i/A HusbandOf( x, z) A W oman(z) A Husbandry, z). 
From (4)': x = y\! ~<God(x ) V ->God(y). 



34 



Roberts 1992, p. 645. 
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Depending on whether variables are written in an evenly or oddly enclosed area, 
the last step tells us how to make sentences out of these wffs. And, the readings 
for these four graphs are the following: 

For (1): T A 3x3yx /j/AT. 

(I.e. At least two things exist.) 

For (2): T A (3 x\/yx = y V -iT). 

(I.e. Something is identical with everything.) 

For (3): 3x3y(x ^ y A 3z[HusbandOf(x, z) A Woman(z) A Husbandry, z)]). 

(I.e. Some woman has at least two husbands.) 

For (4): \/x\/y[x = y V ~^God(x) V ~^God(y)\. 

(I.e. At most one God exists.) 

6 Conclusion 

We have explored a new reading algorithm of Peirce’s Beta graphs. This project 
suggests the importance of several aspects of graphic representation systems. 
When we read off more fine-grained visual features present in graphs we obtain 
an easier and more direct reading method. The identity among individuals is 
visualized by one and the same LI in EG, while different tokens of the same type 
of variable are introduced in first-order languages. Also, a distinction between 
universal and existential statements is made not by different syntactic devices 
but by different ways to read off a graph. A similar distinction is made between 
conjunction and disjunction. Different ways of carving up a graph can give us 
different, but logically equivalent, translations. By contrast, no symbolic system 
allows us to carve up a given sentence in more than one way. 

Two of the main reasons why these visual features have been ignored are, I 
believe, our biased attitude for symbolic systems and unclear distinctions among 
different forms of representation systems. When the logical equivalence between 
a symbolic system and a graphic system is established, many have thought that 
our investigation of the graphic system is accomplished. Hence, we are left with 
the wrong impression that graphic systems can become, at best, as logically 
powerful as symbolic systems. 

On the other hand, we make a distinction among different symbolic logical 
systems. For example, an axiomatic system is chosen for the investigation of 
logical theories and a natural deductive system for the purpose of deduction. 
Similarly, if we can articulate the differences between symbolic and graphic sys- 
tems, we may sometimes choose one over the other depending on our purpose. 
All this suggests that our investigation of graphic systems should go further than 
the proof of the logical equivalence between symbolic and graphic systems. This 
work illustrates one of the many steps we could take for further investigation of 
graphic representation. 

By no means do I argue that EG is better than first-order languages. On 
the contrary, I do not believe in an absolute comparison among different logical 
systems. All I present in this limited space is a case study in approaching one 
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graphic system based on its own strengths, not based on the criteria used for 
symbolic systems. I hope this approach will inspire others not only to find further 
visual aspects of Peirce’s EG and other graphic systems, but also to find more 
concrete differences between graphic and symbolic systems. In this way, we will 
be able to identify different strengths of different kinds of representation systems 
so that we may take full advantage of these differences in research on the multi- 
modal reasoning. 
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Abstract. This paper examines diagrams which exploit qualitative spa- 
tial relations (QSRs) for representation. Our point of departure is the 
theory that such diagram systems are most effective when their formal 
properties match those of the domains that they represent (e.g. [1,2,3]). 
We argue that this is true in certain cases (e.g. when a user is constructing 
diagrammatic representations of a certain kind) but that formal prop- 
erties cannot be studied in isolation from an account of the cognitive 
capacities of diagram users to detect and categorize diagram objects and 
relations. 

We discuss a cognitively salient repertoire of elements in qualitative vi- 
sual languages, which is different from the set of primitives in mathe- 
matical topology, and explore how this repertoire affects the expressivity 
of the languages in terms of their vocabulary and the possible spatial 
relations between diagram elements. 

We then give a detailed analysis of the formal properties of relations 
between the diagram elements. It is shown that the analysis can be ex- 
ploited systematically for the purposes of designing a diagram system 
and analysing expressivity. We demonstrate this methodology with ref- 
erence to several domains, e.g. diagrams for file systems and set theory 
(see e.g. [4]). 



1 Introduction 

This paper examines diagrams that rely on qualitative spatial relations (QSRs) 
- such as regions overlapping or lines intersecting - to stand for relations in a 
domain (i.e. the structure which is represented by the diagrams). We distinguish 
this class of diagrams from those which employ quantitative spatial relations; for 
example where a difference in size between different graphical objects is mean- 
ingful. Some practical examples of this class of diagram system are: UML [5], 
category theory, statecharts [6], flowcharts, file diagrams, topological maps, Euler 
Circles [7], and Venn diagrams. 
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Our objectives with this paper are to; 

— further the programme of analysis of the expressive power of diagrams which 
employ QSRs; 

— draw attention to the central role of cognitive factors in such a theory; 

— propose a “constraint matching” methodology for designing effective diagram 
systems which employ QSRs (see section 2). 

We will illustrate how our analysis can be used for design and evaluation with 
reference to some well-known domains and representation systems (directory 
tree representations, linear diagrams, web site maps, set-theory). 

The formal study of diagrams is a relatively recent phenomenon. Current 
studies suggest a focus on structural properties of spatial relations, arguing that 
diagrams are more useful if the mathematical properties of the visual represen- 
tations match those of the domain [1,2,3]. A particular instance of this argument 
is where formal properties of the visual language collude to restrict possible di- 
agrammatic representations in such a way that impossible states of the domain 
cannot be represented (e.g. Euler Circles). This is considered to be of benefit in 
situations where users construct their own diagrams for problem solving (it is 
known as “self-consistency” [ .]). 

This paper is a development and exploration of the formal perspective on di- 
agrams introduced by [1,2,3,4,7,8,9,10,11]. The reader is referred to these papers 
for definitions and explanations of concepts such as isomorphic representation, 
soundness, correctness, self-consistency, and free-rides. As well as this seman- 
tic emphasis, our approach is distinguished from preceding formal approaches 
(e.g. [12,13]) in that we focus on a cognitively relevant vocabulary of graphi- 
cal primitives (without translation into a spatial logic) and that we pay special 
attention to spatial constraints on diagrams such as planarity. 

We recognize the formal perspective as a productive approach to a better 
understanding of diagrams; one which we can use to aid diagram design [14,15]. 
However, we also argue that concentrating solely on formal properties of spatial 
relations overlooks a non-trivial cognitive step. In order for the diagram user 
to be able to take advantage of a correspondence between formal properties of 
the domain and those of the visual language in general or of “free-rides” in 
particular, they first need to perform category detection and identification. 

Being able to detect a spatial relation in a diagram and match it to a relation 
in the domain involves category detection and identification by the user: they 
need to identify categories of spatial objects in the visual representation and 
match them to categories in the domain. We argue that such processes, as well 
as formal properties of the relations, place restrictions on the expressivity of 
diagram systems. 

2 Background 

2.1 Scope and Terminology 

The subject of this paper is diagrams which employ QUALITATIVE spatial rela- 
tions (QSRs). For example, overlap is a binary spatial relation. If we consider 
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overlap as a qualitative relation, we are concerned with whether any pair of re- 
gions overlap, but we are not concerned with comparing the amount of overlap 
between regions. Our objective is to begin a systematic categorization of possi- 
ble QSRs for the purpose of aiding the process of designing a visual language 
for a particular domain. We will refer to a visual language that relies on QSRs 
as a qualitative visual language or QVL. A QVL consists of a repertoire OF 
GRAPHEMES, and a set of qualitative spatial relations that can hold between 
them. A grapheme is a spatial object (following [16]); the repertoire of graphemes 
is a set of types of spatial objects (more on this below). In order to distinguish 
lines (very narrow spatial tokens) from other shapes, we will also introduce the 
term blob for shapes that are not lines or points. 

2.2 Qualitative Spatial Relations vs. Topology 

Formal topology is one useful tool for examining QSRs — results from topology 
have very direct consequences for many types of QVL, as will become apparent in 
section 5.2. However, it is important to realise that from the perspective of con- 
structing diagrams, the starting point for establishing a repertoire of graphemes 
and the relations between them is not abstract mathematical theory, but the 
cognitive capacities of diagram users. Fundamental to the design of visual lan- 
guages is human categorisation of spatial objects and our ability to distinguish 
relations between these objects (the available repertoire of graphemes and 
relations). Less fundamental, but nevertheless important for usability reasons 
is our ability to detect patterns and differences in a display that contains many 
tokens (scalability). 

3 Graphemes 

3.1 Repertoire 

As noted in section 2, formal topology is a useful tool for analysing spatial 
relations. QVLs in general, however, typically rely on a different set of spatial 
primitives than those used in topological analysis. Several visual relations do not 
have a corresponding topological concept, or can only be described in ways that 
are difficult to understand or unintuitive to the lay person. 

Topology formally recognises two types of objects: regions and points. For 
humans, the distinction between lines and blobs appears to be more intuitive 
or basic. That is, it is common for QVLs to exploit the distinction between 
different types of blob on the one hand, and different types of line on the other, 
to correspond to a fundamental category difference in the represented domain. 
For instance, file system and web site diagrams typically use blobs to stand for 
tokens in the domain (files and directories) , and lines to denote (possibly different 
types of) relations between them. However, there is no easy way to distinguish 
between lines and blobs in topology (both are connected sets of points) . 

We argue that the distinction between line and blob is the most fundamental 
one with respect to graphemes in visual languages. These are the primitives, 
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which stand for the primary categories recognized. In contrast, individual points 
(in the topological sense) are not available as primary objects from this cognitive 
design perspective. While they can be represented implicitly, as a point inside 
a region or as the point where two lines cross, they are difficult to represent 
explicitly. Points can be used as abstractions in the process of interpreting a 
diagram, but they are not primitives of the visual language. 

Although we have no experimental evidence for this claim, it seems likely 
that there is a natural tendency to (attempt to) correlate perceived categories 
of graphical tokens with perceived categories in the represented domain . 1 For 
example, if a particular language uses lines to denote relations between objects, 
it would be confusing to introduce another type of line to stand for an object. 
It seems likely that to maximize usability, the representation system ought to 
correlate graphical types that tend to be perceived as the same category (e.g. dif- 
ferent types of line) with domain types that are also perceived to belong to the 
same category. 



3.2 Scalability 

One of the issues that needs to be considered when designing a representation 
system is how much domain information can be represented in a single view. 
Here, matching categories in a visual language to categories in the represented 
domain can become problematic. Unless the objects in the domain have an obvi- 
ous visual representation (e.g. different types of animals), arbitrary shapes must 
be used, and in that case, achieving category correlation can be difficult. 

A diagram designer can create visually recognizable categories of graphemes 
by using shape, color, pattern and color saturation, or by adding annotation 
such as mini-icons, arrowheads, or kinks on lines. But compared with textual 
languages this strategy is only available for a limited set of categories, since there 
are cognitive limits on the number of visual categories that can be effectively 
recognised. 

If the distinguishing features are not iconic, then the diagram user must 
maintain the meanings of the annotations (e.g. “blue” means functional naviga- 
tion link, “red” means broken link, etc.) in working memory (or else a key must 
be supplied and frequently referenced). This all adds to the effort of processing 
the diagram. Indeed, this shows that graphical representations may involve a 
type of processing that does arise with textual representations, where it is often 
possible to use mnemonic names. 

The issue of preserving category structure is clearly recognized by application 
designers, as there are some common strategies for helping the user perform 
categorization tasks. One such strategy is to allow filtering, whereby the user 
can choose to hide graphemes representing domain objects that belong to certain 

1 A category in the represented domain could be a class of objects, a class of relations, 
or some other feature, like time. 



